2024-04-26T20:59:35.053 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T20:59:35.054 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_rishabh@teuthology --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/orig.config.yaml 2024-04-26T20:59:35.105 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:59:35.200 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7675202' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.dir_split install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1qm62gAeYkestPhYWVTQBiHq+iPVL+FdE0Ka7yEjBn/mC6eyvlkNGLoq2oQvW8Iil6GrJ00obRgIKhML+o8Cw= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEbGvoeTaHyr4/ixoX5P2e2MpvSQEAfur0jKtC7NJam6uWziuqP2WpYmHDAGZRaupIUq3X+n3G2bJSqPrhYJFk= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDb1rZTQQ2O2itzxFRXWiq7qAXZ2rPLKnJ27CS6pwIhirhsypeLElHP6ZDNb71EocuG6+21n0bIgx87+0o8WToc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T20:59:35.200 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T20:59:35.201 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T20:59:35.201 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T20:59:35.203 INFO:teuthology.task.internal:Checking packages... 2024-04-26T20:59:35.227 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T20:59:35.227 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T20:59:35.227 INFO:teuthology.packaging:ref: None 2024-04-26T20:59:35.227 INFO:teuthology.packaging:tag: None 2024-04-26T20:59:35.227 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T20:59:35.227 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:59:35.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T20:59:35.369 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T20:59:35.370 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T20:59:35.377 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T20:59:35.377 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T20:59:35.444 INFO:teuthology.task.internal:Saving configuration 2024-04-26T20:59:35.466 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T20:59:35.524 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:59:35.550 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:54:28.637144', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1qm62gAeYkestPhYWVTQBiHq+iPVL+FdE0Ka7yEjBn/mC6eyvlkNGLoq2oQvW8Iil6GrJ00obRgIKhML+o8Cw='} 2024-04-26T20:59:35.573 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:54:28.635541', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEbGvoeTaHyr4/ixoX5P2e2MpvSQEAfur0jKtC7NJam6uWziuqP2WpYmHDAGZRaupIUq3X+n3G2bJSqPrhYJFk='} 2024-04-26T20:59:35.596 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:54:28.638519', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDb1rZTQQ2O2itzxFRXWiq7qAXZ2rPLKnJ27CS6pwIhirhsypeLElHP6ZDNb71EocuG6+21n0bIgx87+0o8WToc='} 2024-04-26T20:59:35.596 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T20:59:35.603 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T20:59:35.603 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T20:59:35.603 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T20:59:35.603 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T20:59:35.799 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9833c6b160>, signals=[15]) 2024-04-26T20:59:35.800 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T20:59:35.806 INFO:teuthology.task.internal:Opening connections... 2024-04-26T20:59:35.806 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-04-26T20:59:35.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:59:35.884 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-04-26T20:59:35.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:59:35.958 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-04-26T20:59:35.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:59:36.033 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T20:59:36.039 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-04-26T20:59:36.057 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-04-26T20:59:36.058 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:59:36.115 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:59:36.116 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:59:36.116 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:59:36.116 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:59:36.116 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:59:36.116 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-04-26T20:59:36.145 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-04-26T20:59:36.163 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-04-26T20:59:36.163 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-04-26T20:59:36.219 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:59:36.220 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:59:36.221 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-04-26T20:59:36.263 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-04-26T20:59:36.281 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-04-26T20:59:36.281 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-04-26T20:59:36.338 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-04-26T20:59:36.338 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:59:36.339 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:59:36.340 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-04-26T20:59:36.369 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T20:59:36.381 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T20:59:36.428 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T20:59:36.428 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:59:36.431 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:59:36.433 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:59:36.449 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T20:59:36.471 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T20:59:36.472 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:59:36.488 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:59:36.490 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:59:36.602 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T20:59:36.667 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T20:59:36.680 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T20:59:36.680 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-26T20:59:36.680 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-26T20:59:36.680 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:59:36.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:59:36.697 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-04-26T20:59:36.754 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:59:36.754 INFO:teuthology.task.kernel:Running kernel on smithi129: 5.14.0-437.el9.x86_64 2024-04-26T20:59:36.754 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum install -y kernel 2024-04-26T20:59:39.042 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T20:59:43.443 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 19 MB 00:03 2024-04-26T20:59:47.946 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.2 MB 00:00 2024-04-26T20:59:49.602 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T20:59:55.837 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 21 MB 00:06 2024-04-26T21:00:00.953 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-26T21:00:02.834 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T21:00:02.835 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T21:00:02.926 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-26T21:00:02.927 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-04-26T21:00:02.927 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-04-26T21:00:03.038 DEBUG:teuthology.orchestra.run.smithi129:> echo no | sudo yum reinstall kernel || true 2024-04-26T21:00:03.544 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:00:00 PM UTC. 2024-04-26T21:00:03.664 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout:Reinstalling: 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 255 k 2024-04-26T21:00:03.686 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 0 2024-04-26T21:00:03.687 INFO:teuthology.orchestra.run.smithi129.stderr:Operation aborted. 2024-04-26T21:00:03.736 INFO:teuthology.orchestra.run.smithi129.stdout:Is this ok [y/N]: 2024-04-26T21:00:03.736 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum reinstall -y kernel || true 2024-04-26T21:00:04.230 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 09:00:00 PM UTC. 2024-04-26T21:00:04.347 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-26T21:00:04.347 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-26T21:00:04.347 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout:Reinstalling: 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 255 k 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 0 2024-04-26T21:00:04.348 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-04-26T21:00:04.755 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-26T21:00:04.755 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:00:04.755 INFO:teuthology.orchestra.run.smithi129.stdout:Total 626 kB/s | 255 kB 00:00 2024-04-26T21:00:04.771 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-04-26T21:00:04.960 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-04-26T21:00:04.960 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-04-26T21:00:04.965 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-04-26T21:00:04.965 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-04-26T21:00:05.103 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-04-26T21:00:05.172 INFO:teuthology.orchestra.run.smithi129.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:00:05.173 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:00:05.808 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:00:05.809 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:00:06.512 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:00:06.512 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:00:06.512 INFO:teuthology.orchestra.run.smithi129.stdout:Reinstalled: 2024-04-26T21:00:06.512 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T21:00:06.512 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:00:06.512 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-04-26T21:00:06.683 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T21:00:06.771 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T21:00:06.771 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T21:00:06.771 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T21:00:06.772 DEBUG:teuthology.parallel:result is None 2024-04-26T21:00:06.772 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T21:00:06.785 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T21:00:06.785 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:00:06.788 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:00:06.790 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:00:06.808 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T21:00:06.823 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T21:00:06.900 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T21:00:06.900 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:00:06.902 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:00:06.905 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:00:07.024 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T21:00:07.031 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T21:00:07.031 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:00:07.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:00:07.048 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:00:07.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:00:07.065 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:00:07.067 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:00:07.090 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:00:07.109 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:00:07.121 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:00:07.128 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:00:07.149 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:00:07.154 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:00:07.192 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:00:07.194 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T21:00:07.201 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T21:00:07.201 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:00:07.204 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:00:07.206 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:00:07.233 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T21:00:07.250 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T21:00:07.250 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:00:07.268 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:00:07.273 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:00:07.289 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:00:07.363 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:00:07.413 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:00:07.497 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:00:07.534 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:00:07.534 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:00:07.597 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:00:07.643 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:00:07.684 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:00:07.770 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:00:07.812 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:00:07.812 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:00:07.878 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:00:07.924 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:00:07.968 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:00:08.054 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:00:08.093 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:00:08.093 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:00:08.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-04-26T21:00:08.161 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-04-26T21:00:08.163 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-04-26T21:00:08.191 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:00:08.198 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:00:08.237 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:00:08.670 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T21:00:08.680 INFO:teuthology.task.internal:Starting timer... 2024-04-26T21:00:08.680 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T21:00:08.721 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T21:00:08.781 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T21:00:08.781 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-04-26T21:00:08.842 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-04-26T21:00:08.844 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-04-26T21:00:08.905 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-04-26T21:00:08.907 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-04-26T21:00:08.973 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-04-26T21:00:08.975 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T21:00:08.976 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-04-26T21:00:09.000 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-04-26T21:00:09.001 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-04-26T21:00:09.028 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-04-26T21:00:09.028 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-04-26T21:00:09.069 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-04-26T21:00:09.069 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-04-26T21:00:09.069 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:00:09.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:00:09.094 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:00:09.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:00:09.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:00:09.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:00:09.149 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T21:00:09.149 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:00:09.173 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:00:09.199 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:00:09.225 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T21:00:09.235 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T21:00:09.252 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T21:00:09.254 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi129.front.sepia.ceph.com,smithi150.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:05:10.290 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-04-26T21:05:10.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-26T21:05:10.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:05:10.369 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-04-26T21:05:10.453 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-26T21:05:10.453 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-26T21:05:10.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:05:10.529 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-04-26T21:05:10.574 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-26T21:05:10.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-26T21:05:10.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:05:10.651 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-04-26T21:05:10.734 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-26T21:05:10.735 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:05:10.745 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:05:10.745 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:05:10.746 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:05:10.748 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:05:10.748 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:05:10.751 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:05:10.751 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:05:10.775 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:05:10.779 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:05:10.788 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:05:10.794 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:05:10.804 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:05:10.811 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-04-26T21:05:10.817 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:05:10.818 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-04-26T21:05:10.821 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-04-26T21:05:10.829 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-04-26T21:05:10.834 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:05:10.840 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-04-26T21:05:10.843 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:05:10.847 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:05:10.851 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-04-26T21:05:10.856 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:05:10.864 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:05:10.877 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:05:10.901 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:05:10.905 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:05:10.905 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-04-26T21:05:10.905 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.905 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.905 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.905 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.912 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:05:10.915 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:05:10.915 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-26T21:05:10.915 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.915 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.916 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.916 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.935 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:05:10.939 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:05:10.939 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-26T21:05:10.939 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.939 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.939 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.939 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:10.940 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:05:10.949 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:05:10.949 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:05:10.949 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:05:10.949 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:05:10.954 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:05:10.955 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:05:10.955 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:05:10.955 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:05:10.957 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:05:10.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:05:10.958 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:05:10.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:05:11.108 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:05:11.108 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:05:11.176 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:05:11.177 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:05:11.187 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:05:11.187 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:05:11.279 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:05:11.279 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:05:11.279 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:05:11.284 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:05:11.285 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:05:11.285 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:05:11.303 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:05:11.303 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:05:11.303 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:05:11.305 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:05:11.305 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:05:11.309 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:05:11.309 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:05:11.328 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:05:11.328 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:05:11.371 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:05:11.373 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:05:11.393 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:05:11.445 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:05:11.447 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:05:11.466 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:05:11.509 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-04-26T21:05:11.510 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-04-26T21:05:11.511 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-04-26T21:05:11.511 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-04-26T21:05:11.529 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-04-26T21:05:11.530 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-04-26T21:05:11.735 INFO:teuthology.orchestra.run.smithi129.stdout:48 files removed 2024-04-26T21:05:11.736 INFO:teuthology.orchestra.run.smithi150.stdout:48 files removed 2024-04-26T21:05:11.754 INFO:teuthology.orchestra.run.smithi171.stdout:48 files removed 2024-04-26T21:05:11.762 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:05:11.764 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:05:11.782 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:05:12.369 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 256 kB/s | 84 kB 00:00 2024-04-26T21:05:12.433 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 206 kB/s | 84 kB 00:00 2024-04-26T21:05:12.477 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 184 kB/s | 84 kB 00:00 2024-04-26T21:05:12.854 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-26T21:05:13.034 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-04-26T21:05:13.052 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-26T21:05:13.296 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-26T21:05:13.359 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-26T21:05:13.490 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-26T21:05:13.923 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T21:05:14.059 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T21:05:14.095 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T21:05:15.827 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-26T21:05:16.071 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-26T21:05:20.087 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.2 MB 00:00 2024-04-26T21:05:20.272 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-26T21:05:21.704 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-26T21:05:21.915 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-26T21:05:25.016 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 21 MB 00:02 2024-04-26T21:05:27.763 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 21 MB 00:05 2024-04-26T21:05:30.057 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-26T21:05:32.167 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:32.167 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:32.178 INFO:teuthology.orchestra.run.smithi129.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:05:32.178 INFO:teuthology.orchestra.run.smithi129.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:05:32.179 INFO:teuthology.orchestra.run.smithi129.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:05:32.180 INFO:teuthology.orchestra.run.smithi129.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:05:32.182 INFO:teuthology.orchestra.run.smithi129.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:05:32.184 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:05:32.184 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:05:32.244 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-26T21:05:32.254 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-26T21:05:32.254 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-04-26T21:05:32.254 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-26T21:05:32.254 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-04-26T21:05:32.255 INFO:teuthology.orchestra.run.smithi129.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:05:32.255 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:05:32.255 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:32.255 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:05:32.255 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:05:32.255 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:05:32.255 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:05:32.255 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:05:32.255 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:05:32.256 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:05:32.256 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:05:32.256 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:05:32.256 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:05:32.256 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:05:32.256 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:05:32.256 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:05:32.257 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:05:32.257 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:05:32.257 INFO:teuthology.orchestra.run.smithi129.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:05:32.257 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:05:32.257 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:05:32.257 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:05:32.257 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:05:32.257 INFO:teuthology.orchestra.run.smithi129.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:05:32.258 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:05:32.258 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:05:32.258 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:05:32.258 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:05:32.258 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:05:32.258 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:05:32.258 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:05:32.258 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:05:32.259 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:05:32.259 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:05:32.259 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:32.259 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:05:32.259 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:05:32.259 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-04-26T21:05:32.259 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:05:32.259 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:32.260 INFO:teuthology.orchestra.run.smithi129.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:05:32.260 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:05:32.260 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-04-26T21:05:32.260 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:05:32.260 INFO:teuthology.orchestra.run.smithi129.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:05:32.260 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:05:32.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:05:32.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:05:32.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:05:32.261 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:32.262 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:05:32.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:05:32.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:05:32.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:05:32.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:05:32.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:05:32.268 INFO:teuthology.orchestra.run.smithi129.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:05:32.269 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:05:32.269 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:05:32.269 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-04-26T21:05:32.269 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:05:32.269 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:05:32.269 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-04-26T21:05:32.269 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-26T21:05:32.270 INFO:teuthology.orchestra.run.smithi129.stdout:Install 148 Packages 2024-04-26T21:05:32.270 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 4 Packages 2024-04-26T21:05:32.270 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:05:32.270 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 222 M 2024-04-26T21:05:32.271 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-04-26T21:05:32.809 INFO:teuthology.orchestra.run.smithi129.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-26T21:05:33.025 INFO:teuthology.orchestra.run.smithi129.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 4.0 MB/s | 889 kB 00:00 2024-04-26T21:05:33.084 INFO:teuthology.orchestra.run.smithi129.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.4 MB/s | 145 kB 00:00 2024-04-26T21:05:33.096 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 5.1 kB/s | 1.7 kB 00:00 2024-04-26T21:05:33.359 INFO:teuthology.orchestra.run.smithi129.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-04-26T21:05:33.551 INFO:teuthology.orchestra.run.smithi129.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 7.9 MB/s | 1.5 MB 00:00 2024-04-26T21:05:33.855 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:18 2024-04-26T21:05:34.176 INFO:teuthology.orchestra.run.smithi129.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 7.5 MB/s | 4.7 MB 00:00 2024-04-26T21:05:34.243 INFO:teuthology.orchestra.run.smithi129.stdout:(7/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-04-26T21:05:34.452 INFO:teuthology.orchestra.run.smithi129.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 12 MB/s | 21 MB 00:01 2024-04-26T21:05:34.518 INFO:teuthology.orchestra.run.smithi129.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 379 kB/s | 25 kB 00:00 2024-04-26T21:05:35.240 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:35.240 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:35.252 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:05:35.252 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:05:35.253 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:05:35.254 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:05:35.256 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:05:35.258 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:05:35.259 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:05:35.319 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-26T21:05:35.332 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:05:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:05:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:05:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:05:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:05:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:05:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:05:35.336 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-04-26T21:05:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:05:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:05:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:05:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:05:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:05:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:05:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:05:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:05:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:05:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:05:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:05:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:05:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:05:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:05:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:05:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:05:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:05:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:05:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:05:35.346 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-26T21:05:35.346 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-26T21:05:35.346 INFO:teuthology.orchestra.run.smithi150.stdout:Install 148 Packages 2024-04-26T21:05:35.346 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 4 Packages 2024-04-26T21:05:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:05:35.346 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 222 M 2024-04-26T21:05:35.346 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-26T21:05:36.077 INFO:teuthology.orchestra.run.smithi129.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 8.9 MB/s | 17 MB 00:01 2024-04-26T21:05:36.127 INFO:teuthology.orchestra.run.smithi129.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 4.4 MB/s | 225 kB 00:00 2024-04-26T21:05:36.211 INFO:teuthology.orchestra.run.smithi129.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 325 kB/s | 27 kB 00:00 2024-04-26T21:05:36.286 INFO:teuthology.orchestra.run.smithi129.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 423 kB/s | 32 kB 00:00 2024-04-26T21:05:36.445 INFO:teuthology.orchestra.run.smithi129.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.6 MB/s | 747 kB 00:00 2024-04-26T21:05:36.528 INFO:teuthology.orchestra.run.smithi129.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 163 kB 00:00 2024-04-26T21:05:36.645 INFO:teuthology.orchestra.run.smithi129.stdout:(16/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 4.4 MB/s | 11 MB 00:02 2024-04-26T21:05:36.670 INFO:teuthology.orchestra.run.smithi129.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 895 kB/s | 127 kB 00:00 2024-04-26T21:05:36.714 INFO:teuthology.orchestra.run.smithi150.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-26T21:05:36.812 INFO:teuthology.orchestra.run.smithi129.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 2.9 MB/s | 501 kB 00:00 2024-04-26T21:05:36.929 INFO:teuthology.orchestra.run.smithi129.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 387 kB/s | 45 kB 00:00 2024-04-26T21:05:37.046 INFO:teuthology.orchestra.run.smithi129.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.1 MB/s | 135 kB 00:00 2024-04-26T21:05:37.172 INFO:teuthology.orchestra.run.smithi129.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.3 MB/s | 161 kB 00:00 2024-04-26T21:05:37.439 INFO:teuthology.orchestra.run.smithi129.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 7.0 MB/s | 5.4 MB 00:00 2024-04-26T21:05:37.781 INFO:teuthology.orchestra.run.smithi150.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 834 kB/s | 889 kB 00:01 2024-04-26T21:05:37.856 INFO:teuthology.orchestra.run.smithi129.stdout:(23/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 15 MB/s | 49 MB 00:03 2024-04-26T21:05:37.881 INFO:teuthology.orchestra.run.smithi129.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 674 kB/s | 298 kB 00:00 2024-04-26T21:05:37.906 INFO:teuthology.orchestra.run.smithi129.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 99 kB 00:00 2024-04-26T21:05:37.915 INFO:teuthology.orchestra.run.smithi150.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.1 MB/s | 145 kB 00:00 2024-04-26T21:05:37.931 INFO:teuthology.orchestra.run.smithi129.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-26T21:05:37.998 INFO:teuthology.orchestra.run.smithi129.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-26T21:05:38.057 INFO:teuthology.orchestra.run.smithi129.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 588 kB/s | 34 kB 00:00 2024-04-26T21:05:38.082 INFO:teuthology.orchestra.run.smithi129.stdout:(29/152): python3-rados-19.0.0-3306.gbc612ffb.e 353 kB/s | 321 kB 00:00 2024-04-26T21:05:38.108 INFO:teuthology.orchestra.run.smithi129.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.7 MB/s | 142 kB 00:00 2024-04-26T21:05:38.158 INFO:teuthology.orchestra.run.smithi129.stdout:(31/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 12 MB/s | 3.1 MB 00:00 2024-04-26T21:05:38.217 INFO:teuthology.orchestra.run.smithi129.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 4.2 MB/s | 249 kB 00:00 2024-04-26T21:05:38.259 INFO:teuthology.orchestra.run.smithi129.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 1.2 MB/s | 49 kB 00:00 2024-04-26T21:05:38.301 INFO:teuthology.orchestra.run.smithi129.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 396 kB/s | 16 kB 00:00 2024-04-26T21:05:38.317 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-04-26T21:05:38.359 INFO:teuthology.orchestra.run.smithi129.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 4.6 MB/s | 273 kB 00:00 2024-04-26T21:05:38.443 INFO:teuthology.orchestra.run.smithi129.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 8.9 MB/s | 762 kB 00:00 2024-04-26T21:05:38.493 INFO:teuthology.orchestra.run.smithi129.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 366 kB/s | 18 kB 00:00 2024-04-26T21:05:38.593 INFO:teuthology.orchestra.run.smithi129.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 806 kB/s | 80 kB 00:00 2024-04-26T21:05:38.702 INFO:teuthology.orchestra.run.smithi129.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-26T21:05:38.786 INFO:teuthology.orchestra.run.smithi129.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-26T21:05:38.836 INFO:teuthology.orchestra.run.smithi129.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-26T21:05:38.869 INFO:teuthology.orchestra.run.smithi129.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-26T21:05:38.920 INFO:teuthology.orchestra.run.smithi129.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-26T21:05:38.945 INFO:teuthology.orchestra.run.smithi129.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-26T21:05:38.970 INFO:teuthology.orchestra.run.smithi129.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-26T21:05:38.995 INFO:teuthology.orchestra.run.smithi129.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T21:05:39.021 INFO:teuthology.orchestra.run.smithi129.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-26T21:05:39.087 INFO:teuthology.orchestra.run.smithi129.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-26T21:05:39.121 INFO:teuthology.orchestra.run.smithi129.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-04-26T21:05:39.197 INFO:teuthology.orchestra.run.smithi129.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 984 kB/s | 107 kB 00:00 2024-04-26T21:05:39.255 INFO:teuthology.orchestra.run.smithi129.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-26T21:05:39.306 INFO:teuthology.orchestra.run.smithi129.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-04-26T21:05:39.398 INFO:teuthology.orchestra.run.smithi150.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-04-26T21:05:39.414 INFO:teuthology.orchestra.run.smithi129.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-26T21:05:39.456 INFO:teuthology.orchestra.run.smithi129.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-04-26T21:05:39.481 INFO:teuthology.orchestra.run.smithi129.stdout:(55/152): elfutils-libelf-devel-0.190-2.el9.x86 60 kB/s | 21 kB 00:00 2024-04-26T21:05:39.506 INFO:teuthology.orchestra.run.smithi129.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-26T21:05:39.532 INFO:teuthology.orchestra.run.smithi129.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T21:05:39.590 INFO:teuthology.orchestra.run.smithi129.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-26T21:05:39.616 INFO:teuthology.orchestra.run.smithi129.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-04-26T21:05:39.649 INFO:teuthology.orchestra.run.smithi129.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-04-26T21:05:39.674 INFO:teuthology.orchestra.run.smithi129.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-26T21:05:39.708 INFO:teuthology.orchestra.run.smithi129.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-26T21:05:39.758 INFO:teuthology.orchestra.run.smithi129.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-26T21:05:39.809 INFO:teuthology.orchestra.run.smithi129.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-04-26T21:05:39.881 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-26T21:05:39.907 INFO:teuthology.orchestra.run.smithi150.stdout:(5/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 5.8 MB/s | 21 MB 00:03 2024-04-26T21:05:39.926 INFO:teuthology.orchestra.run.smithi129.stdout:(65/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 1.9 MB/s | 3.5 MB 00:01 2024-04-26T21:05:39.993 INFO:teuthology.orchestra.run.smithi129.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-26T21:05:40.060 INFO:teuthology.orchestra.run.smithi129.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-26T21:05:40.085 INFO:teuthology.orchestra.run.smithi129.stdout:(68/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 713 kB/s | 292 kB 00:00 2024-04-26T21:05:40.127 INFO:teuthology.orchestra.run.smithi129.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-04-26T21:05:40.194 INFO:teuthology.orchestra.run.smithi129.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-26T21:05:40.240 INFO:teuthology.orchestra.run.smithi150.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-26T21:05:40.283 INFO:teuthology.orchestra.run.smithi150.stdout:(7/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-04-26T21:05:40.311 INFO:teuthology.orchestra.run.smithi129.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-04-26T21:05:40.344 INFO:teuthology.orchestra.run.smithi129.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-04-26T21:05:40.369 INFO:teuthology.orchestra.run.smithi129.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T21:05:40.428 INFO:teuthology.orchestra.run.smithi129.stdout:(74/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-04-26T21:05:40.453 INFO:teuthology.orchestra.run.smithi129.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-26T21:05:40.475 INFO:teuthology.orchestra.run.smithi150.stdout:(8/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.3 MB/s | 4.7 MB 00:00 2024-04-26T21:05:40.478 INFO:teuthology.orchestra.run.smithi129.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-26T21:05:40.504 INFO:teuthology.orchestra.run.smithi129.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-26T21:05:40.508 INFO:teuthology.orchestra.run.smithi150.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 748 kB/s | 25 kB 00:00 2024-04-26T21:05:40.529 INFO:teuthology.orchestra.run.smithi129.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-26T21:05:40.562 INFO:teuthology.orchestra.run.smithi129.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T21:05:40.588 INFO:teuthology.orchestra.run.smithi129.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-26T21:05:40.638 INFO:teuthology.orchestra.run.smithi129.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-26T21:05:40.722 INFO:teuthology.orchestra.run.smithi129.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-26T21:05:40.747 INFO:teuthology.orchestra.run.smithi129.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-26T21:05:40.772 INFO:teuthology.orchestra.run.smithi129.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T21:05:40.839 INFO:teuthology.orchestra.run.smithi129.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-04-26T21:05:40.865 INFO:teuthology.orchestra.run.smithi129.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-04-26T21:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-04-26T21:05:40.915 INFO:teuthology.orchestra.run.smithi129.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-26T21:05:40.940 INFO:teuthology.orchestra.run.smithi129.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T21:05:40.957 INFO:teuthology.orchestra.run.smithi129.stdout:(90/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-26T21:05:40.982 INFO:teuthology.orchestra.run.smithi129.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 801 kB/s | 54 kB 00:00 2024-04-26T21:05:41.008 INFO:teuthology.orchestra.run.smithi129.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-04-26T21:05:41.033 INFO:teuthology.orchestra.run.smithi129.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-04-26T21:05:41.058 INFO:teuthology.orchestra.run.smithi129.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-26T21:05:41.084 INFO:teuthology.orchestra.run.smithi129.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-04-26T21:05:41.099 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-26T21:05:41.109 INFO:teuthology.orchestra.run.smithi129.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-26T21:05:41.134 INFO:teuthology.orchestra.run.smithi129.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-26T21:05:41.159 INFO:teuthology.orchestra.run.smithi129.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T21:05:41.185 INFO:teuthology.orchestra.run.smithi129.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-26T21:05:41.210 INFO:teuthology.orchestra.run.smithi129.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T21:05:41.235 INFO:teuthology.orchestra.run.smithi129.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-04-26T21:05:41.286 INFO:teuthology.orchestra.run.smithi129.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-26T21:05:41.361 INFO:teuthology.orchestra.run.smithi129.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-04-26T21:05:41.561 INFO:teuthology.orchestra.run.smithi129.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-04-26T21:05:41.828 INFO:teuthology.orchestra.run.smithi129.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2024-04-26T21:05:41.904 INFO:teuthology.orchestra.run.smithi129.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 888 kB/s | 548 kB 00:00 2024-04-26T21:05:42.004 INFO:teuthology.orchestra.run.smithi129.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-26T21:05:42.029 INFO:teuthology.orchestra.run.smithi129.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 482 kB/s | 309 kB 00:00 2024-04-26T21:05:42.071 INFO:teuthology.orchestra.run.smithi129.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 728 kB/s | 49 kB 00:00 2024-04-26T21:05:42.113 INFO:teuthology.orchestra.run.smithi129.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 808 kB/s | 67 kB 00:00 2024-04-26T21:05:42.146 INFO:teuthology.orchestra.run.smithi129.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-26T21:05:42.265 INFO:teuthology.orchestra.run.smithi129.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.5 MB/s | 548 kB 00:00 2024-04-26T21:05:42.332 INFO:teuthology.orchestra.run.smithi129.stdout:(113/152): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-04-26T21:05:42.366 INFO:teuthology.orchestra.run.smithi129.stdout:(114/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-04-26T21:05:42.391 INFO:teuthology.orchestra.run.smithi129.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2024-04-26T21:05:42.419 INFO:teuthology.orchestra.run.smithi129.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 372 kB/s | 32 kB 00:00 2024-04-26T21:05:42.458 INFO:teuthology.orchestra.run.smithi129.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-26T21:05:42.483 INFO:teuthology.orchestra.run.smithi129.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T21:05:42.509 INFO:teuthology.orchestra.run.smithi129.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.9 MB/s | 358 kB 00:00 2024-04-26T21:05:42.542 INFO:teuthology.orchestra.run.smithi129.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-26T21:05:42.568 INFO:teuthology.orchestra.run.smithi129.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-04-26T21:05:42.593 INFO:teuthology.orchestra.run.smithi129.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-26T21:05:42.643 INFO:teuthology.orchestra.run.smithi129.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-26T21:05:42.668 INFO:teuthology.orchestra.run.smithi129.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-26T21:05:42.719 INFO:teuthology.orchestra.run.smithi129.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-26T21:05:42.761 INFO:teuthology.orchestra.run.smithi129.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-04-26T21:05:42.794 INFO:teuthology.orchestra.run.smithi129.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 184 kB/s | 23 kB 00:00 2024-04-26T21:05:42.819 INFO:teuthology.orchestra.run.smithi129.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-26T21:05:42.844 INFO:teuthology.orchestra.run.smithi129.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-26T21:05:42.895 INFO:teuthology.orchestra.run.smithi129.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-26T21:05:42.920 INFO:teuthology.orchestra.run.smithi129.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-04-26T21:05:42.945 INFO:teuthology.orchestra.run.smithi129.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 784 kB/s | 79 kB 00:00 2024-04-26T21:05:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 765 kB/s | 58 kB 00:00 2024-04-26T21:05:43.004 INFO:teuthology.orchestra.run.smithi129.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-26T21:05:43.029 INFO:teuthology.orchestra.run.smithi129.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-26T21:05:43.055 INFO:teuthology.orchestra.run.smithi129.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-26T21:05:43.080 INFO:teuthology.orchestra.run.smithi129.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-26T21:05:43.106 INFO:teuthology.orchestra.run.smithi129.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-26T21:05:43.130 INFO:teuthology.orchestra.run.smithi129.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T21:05:43.156 INFO:teuthology.orchestra.run.smithi129.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-04-26T21:05:43.181 INFO:teuthology.orchestra.run.smithi129.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-26T21:05:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-26T21:05:43.240 INFO:teuthology.orchestra.run.smithi129.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T21:05:43.273 INFO:teuthology.orchestra.run.smithi129.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 975 kB/s | 90 kB 00:00 2024-04-26T21:05:43.307 INFO:teuthology.orchestra.run.smithi129.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-04-26T21:05:43.333 INFO:teuthology.orchestra.run.smithi129.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-04-26T21:05:43.358 INFO:teuthology.orchestra.run.smithi129.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-26T21:05:43.408 INFO:teuthology.orchestra.run.smithi129.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-26T21:05:43.592 INFO:teuthology.orchestra.run.smithi129.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-04-26T21:05:43.684 INFO:teuthology.orchestra.run.smithi129.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-26T21:05:43.776 INFO:teuthology.orchestra.run.smithi129.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 7.7 MB/s | 3.4 MB 00:00 2024-04-26T21:05:43.900 INFO:teuthology.orchestra.run.smithi150.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 2.9 MB/s | 11 MB 00:03 2024-04-26T21:05:44.042 INFO:teuthology.orchestra.run.smithi150.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 225 kB 00:00 2024-04-26T21:05:44.109 INFO:teuthology.orchestra.run.smithi150.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 406 kB/s | 27 kB 00:00 2024-04-26T21:05:44.176 INFO:teuthology.orchestra.run.smithi150.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 477 kB/s | 32 kB 00:00 2024-04-26T21:05:44.209 INFO:teuthology.orchestra.run.smithi129.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-04-26T21:05:44.215 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:05:44.215 INFO:teuthology.orchestra.run.smithi129.stdout:Total 19 MB/s | 222 MB 00:11 2024-04-26T21:05:44.726 INFO:teuthology.orchestra.run.smithi150.stdout:(14/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 12 MB/s | 49 MB 00:04 2024-04-26T21:05:44.760 INFO:teuthology.orchestra.run.smithi150.stdout:(15/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 1.2 MB/s | 747 kB 00:00 2024-04-26T21:05:44.785 INFO:teuthology.orchestra.run.smithi150.stdout:(16/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 163 kB 00:00 2024-04-26T21:05:44.852 INFO:teuthology.orchestra.run.smithi150.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 7.4 MB/s | 501 kB 00:00 2024-04-26T21:05:44.911 INFO:teuthology.orchestra.run.smithi150.stdout:(18/152): librados-devel-19.0.0-3306.gbc612ffb. 844 kB/s | 127 kB 00:00 2024-04-26T21:05:44.986 INFO:teuthology.orchestra.run.smithi150.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 602 kB/s | 45 kB 00:00 2024-04-26T21:05:45.061 INFO:teuthology.orchestra.run.smithi150.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.8 MB/s | 135 kB 00:00 2024-04-26T21:05:45.170 INFO:teuthology.orchestra.run.smithi150.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.5 MB/s | 161 kB 00:00 2024-04-26T21:05:45.316 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-04-26T21:05:45.337 INFO:teuthology.orchestra.run.smithi150.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 11 MB/s | 5.4 MB 00:00 2024-04-26T21:05:45.362 INFO:teuthology.orchestra.run.smithi150.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 321 kB 00:00 2024-04-26T21:05:45.387 INFO:teuthology.orchestra.run.smithi150.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 5.8 MB/s | 298 kB 00:00 2024-04-26T21:05:45.421 INFO:teuthology.orchestra.run.smithi150.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 99 kB 00:00 2024-04-26T21:05:45.446 INFO:teuthology.orchestra.run.smithi150.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T21:05:45.460 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-04-26T21:05:45.460 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-04-26T21:05:45.496 INFO:teuthology.orchestra.run.smithi150.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 3.3 MB/s | 171 kB 00:00 2024-04-26T21:05:45.547 INFO:teuthology.orchestra.run.smithi150.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 688 kB/s | 34 kB 00:00 2024-04-26T21:05:45.731 INFO:teuthology.orchestra.run.smithi150.stdout:(29/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 3.1 MB/s | 17 MB 00:05 2024-04-26T21:05:45.756 INFO:teuthology.orchestra.run.smithi150.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 681 kB/s | 142 kB 00:00 2024-04-26T21:05:46.224 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T21:05:46.289 INFO:teuthology.orchestra.run.smithi150.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 14 MB/s | 7.4 MB 00:00 2024-04-26T21:05:46.339 INFO:teuthology.orchestra.run.smithi150.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 4.9 MB/s | 249 kB 00:00 2024-04-26T21:05:46.398 INFO:teuthology.orchestra.run.smithi150.stdout:(33/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.2 MB/s | 3.1 MB 00:00 2024-04-26T21:05:46.423 INFO:teuthology.orchestra.run.smithi150.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 590 kB/s | 49 kB 00:00 2024-04-26T21:05:46.474 INFO:teuthology.orchestra.run.smithi150.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 5.3 MB/s | 273 kB 00:00 2024-04-26T21:05:46.499 INFO:teuthology.orchestra.run.smithi150.stdout:(36/152): ceph-prometheus-alerts-19.0.0-3306.gb 163 kB/s | 16 kB 00:00 2024-04-26T21:05:46.558 INFO:teuthology.orchestra.run.smithi150.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 8.9 MB/s | 762 kB 00:00 2024-04-26T21:05:46.600 INFO:teuthology.orchestra.run.smithi150.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 182 kB/s | 18 kB 00:00 2024-04-26T21:05:46.775 INFO:teuthology.orchestra.run.smithi150.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 371 kB/s | 80 kB 00:00 2024-04-26T21:05:46.950 INFO:teuthology.orchestra.run.smithi150.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-26T21:05:47.034 INFO:teuthology.orchestra.run.smithi150.stdout:(41/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 2.7 MB/s | 3.5 MB 00:01 2024-04-26T21:05:47.093 INFO:teuthology.orchestra.run.smithi150.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-04-26T21:05:47.126 INFO:teuthology.orchestra.run.smithi150.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-04-26T21:05:47.213 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-04-26T21:05:47.214 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-04-26T21:05:47.235 INFO:teuthology.orchestra.run.smithi150.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 750 kB/s | 106 kB 00:00 2024-04-26T21:05:47.260 INFO:teuthology.orchestra.run.smithi150.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-04-26T21:05:47.319 INFO:teuthology.orchestra.run.smithi150.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-26T21:05:47.402 INFO:teuthology.orchestra.run.smithi150.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-04-26T21:05:47.428 INFO:teuthology.orchestra.run.smithi150.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-04-26T21:05:47.494 INFO:teuthology.orchestra.run.smithi150.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-26T21:05:47.545 INFO:teuthology.orchestra.run.smithi150.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-04-26T21:05:47.562 INFO:teuthology.orchestra.run.smithi150.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-04-26T21:05:47.612 INFO:teuthology.orchestra.run.smithi150.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-04-26T21:05:47.645 INFO:teuthology.orchestra.run.smithi150.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-04-26T21:05:47.687 INFO:teuthology.orchestra.run.smithi150.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-04-26T21:05:47.721 INFO:teuthology.orchestra.run.smithi150.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 337 kB/s | 11 kB 00:00 2024-04-26T21:05:47.780 INFO:teuthology.orchestra.run.smithi150.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-04-26T21:05:47.822 INFO:teuthology.orchestra.run.smithi150.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-26T21:05:47.839 INFO:teuthology.orchestra.run.smithi150.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-26T21:05:47.914 INFO:teuthology.orchestra.run.smithi150.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-04-26T21:05:47.939 INFO:teuthology.orchestra.run.smithi150.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-26T21:05:47.989 INFO:teuthology.orchestra.run.smithi150.stdout:(61/152): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-04-26T21:05:48.015 INFO:teuthology.orchestra.run.smithi150.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-04-26T21:05:48.040 INFO:teuthology.orchestra.run.smithi150.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-04-26T21:05:48.091 INFO:teuthology.orchestra.run.smithi150.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-26T21:05:48.116 INFO:teuthology.orchestra.run.smithi150.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 469 kB/s | 35 kB 00:00 2024-04-26T21:05:48.141 INFO:teuthology.orchestra.run.smithi150.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-26T21:05:48.275 INFO:teuthology.orchestra.run.smithi150.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T21:05:48.343 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:48.343 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:48.354 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:05:48.355 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:05:48.356 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:05:48.356 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:05:48.358 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:05:48.360 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:05:48.361 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:05:48.366 INFO:teuthology.orchestra.run.smithi150.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-04-26T21:05:48.421 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-04-26T21:05:48.437 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-26T21:05:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-04-26T21:05:48.437 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-26T21:05:48.437 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-04-26T21:05:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:05:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:05:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:05:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:05:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:05:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:05:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:05:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:05:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:05:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:05:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:05:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:05:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:05:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:05:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:05:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:05:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:05:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:05:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:05:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:05:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:05:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:05:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:05:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:05:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:05:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:05:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:05:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:05:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:05:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:05:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:05:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:05:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:05:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:05:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:05:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:05:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:05:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:05:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:05:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:05:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:05:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:05:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:05:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:05:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:05:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:05:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:05:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout:Install 148 Packages 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 4 Packages 2024-04-26T21:05:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:05:48.453 INFO:teuthology.orchestra.run.smithi150.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-26T21:05:48.454 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 222 M 2024-04-26T21:05:48.454 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-04-26T21:05:48.500 INFO:teuthology.orchestra.run.smithi150.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-26T21:05:48.534 INFO:teuthology.orchestra.run.smithi150.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-04-26T21:05:48.601 INFO:teuthology.orchestra.run.smithi150.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T21:05:48.660 INFO:teuthology.orchestra.run.smithi150.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-04-26T21:05:48.794 INFO:teuthology.orchestra.run.smithi150.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-26T21:05:48.860 INFO:teuthology.orchestra.run.smithi150.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-04-26T21:05:48.885 INFO:teuthology.orchestra.run.smithi150.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-26T21:05:48.928 INFO:teuthology.orchestra.run.smithi150.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-04-26T21:05:48.953 INFO:teuthology.orchestra.run.smithi150.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 285 kB/s | 19 kB 00:00 2024-04-26T21:05:49.003 INFO:teuthology.orchestra.run.smithi150.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-04-26T21:05:49.037 INFO:teuthology.orchestra.run.smithi150.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-04-26T21:05:49.112 INFO:teuthology.orchestra.run.smithi150.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-26T21:05:49.179 INFO:teuthology.orchestra.run.smithi150.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-26T21:05:49.229 INFO:teuthology.orchestra.run.smithi150.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-04-26T21:05:49.279 INFO:teuthology.orchestra.run.smithi150.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 770 kB/s | 77 kB 00:00 2024-04-26T21:05:49.305 INFO:teuthology.orchestra.run.smithi150.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 561 kB/s | 42 kB 00:00 2024-04-26T21:05:49.371 INFO:teuthology.orchestra.run.smithi150.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-26T21:05:49.422 INFO:teuthology.orchestra.run.smithi150.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-04-26T21:05:49.447 INFO:teuthology.orchestra.run.smithi150.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-26T21:05:49.481 INFO:teuthology.orchestra.run.smithi150.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-26T21:05:49.506 INFO:teuthology.orchestra.run.smithi150.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-04-26T21:05:49.531 INFO:teuthology.orchestra.run.smithi150.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 639 kB/s | 54 kB 00:00 2024-04-26T21:05:49.581 INFO:teuthology.orchestra.run.smithi150.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-04-26T21:05:49.598 INFO:teuthology.orchestra.run.smithi150.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-04-26T21:05:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 522 kB/s | 36 kB 00:00 2024-04-26T21:05:49.692 INFO:teuthology.orchestra.run.smithi150.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-04-26T21:05:49.751 INFO:teuthology.orchestra.run.smithi150.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-04-26T21:05:49.826 INFO:teuthology.orchestra.run.smithi150.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-04-26T21:05:49.851 INFO:teuthology.orchestra.run.smithi150.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 577 kB/s | 58 kB 00:00 2024-04-26T21:05:49.884 INFO:teuthology.orchestra.run.smithi171.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-26T21:05:49.902 INFO:teuthology.orchestra.run.smithi150.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-26T21:05:49.927 INFO:teuthology.orchestra.run.smithi150.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-04-26T21:05:49.978 INFO:teuthology.orchestra.run.smithi150.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-04-26T21:05:50.019 INFO:teuthology.orchestra.run.smithi150.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-04-26T21:05:50.084 INFO:teuthology.orchestra.run.smithi171.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 4.4 MB/s | 889 kB 00:00 2024-04-26T21:05:50.120 INFO:teuthology.orchestra.run.smithi150.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-04-26T21:05:50.134 INFO:teuthology.orchestra.run.smithi171.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.8 MB/s | 145 kB 00:00 2024-04-26T21:05:50.162 INFO:teuthology.orchestra.run.smithi150.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-04-26T21:05:50.195 INFO:teuthology.orchestra.run.smithi150.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 586 kB/s | 19 kB 00:00 2024-04-26T21:05:50.229 INFO:teuthology.orchestra.run.smithi150.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-04-26T21:05:50.313 INFO:teuthology.orchestra.run.smithi150.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 313 kB/s | 26 kB 00:00 2024-04-26T21:05:50.471 INFO:teuthology.orchestra.run.smithi150.stdout:(108/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-04-26T21:05:50.477 INFO:teuthology.orchestra.run.smithi171.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 6.9 MB/s | 2.4 MB 00:00 2024-04-26T21:05:50.496 INFO:teuthology.orchestra.run.smithi150.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 264 kB/s | 49 kB 00:00 2024-04-26T21:05:50.563 INFO:teuthology.orchestra.run.smithi150.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-26T21:05:50.568 INFO:teuthology.orchestra.run.smithi171.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 4.8 MB/s | 5.4 MB 00:01 2024-04-26T21:05:50.589 INFO:teuthology.orchestra.run.smithi150.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-04-26T21:05:50.606 INFO:teuthology.orchestra.run.smithi150.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T21:05:50.639 INFO:teuthology.orchestra.run.smithi150.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-04-26T21:05:50.673 INFO:teuthology.orchestra.run.smithi150.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 965 kB/s | 32 kB 00:00 2024-04-26T21:05:50.698 INFO:teuthology.orchestra.run.smithi150.stdout:(115/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-04-26T21:05:50.724 INFO:teuthology.orchestra.run.smithi150.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-04-26T21:05:50.741 INFO:teuthology.orchestra.run.smithi150.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-04-26T21:05:50.746 INFO:teuthology.orchestra.run.smithi171.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-04-26T21:05:50.766 INFO:teuthology.orchestra.run.smithi150.stdout:(118/152): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-04-26T21:05:50.799 INFO:teuthology.orchestra.run.smithi150.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-26T21:05:50.832 INFO:teuthology.orchestra.run.smithi150.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-26T21:05:50.858 INFO:teuthology.orchestra.run.smithi150.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-04-26T21:05:50.883 INFO:teuthology.orchestra.run.smithi150.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-26T21:05:50.900 INFO:teuthology.orchestra.run.smithi150.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-04-26T21:05:50.917 INFO:teuthology.orchestra.run.smithi150.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 319 kB/s | 11 kB 00:00 2024-04-26T21:05:50.934 INFO:teuthology.orchestra.run.smithi150.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 521 kB/s | 18 kB 00:00 2024-04-26T21:05:50.951 INFO:teuthology.orchestra.run.smithi150.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 681 kB/s | 23 kB 00:00 2024-04-26T21:05:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 580 kB/s | 19 kB 00:00 2024-04-26T21:05:50.985 INFO:teuthology.orchestra.run.smithi150.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 589 kB/s | 20 kB 00:00 2024-04-26T21:05:51.027 INFO:teuthology.orchestra.run.smithi150.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-04-26T21:05:51.044 INFO:teuthology.orchestra.run.smithi150.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 790 kB/s | 46 kB 00:00 2024-04-26T21:05:51.069 INFO:teuthology.orchestra.run.smithi150.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-04-26T21:05:51.086 INFO:teuthology.orchestra.run.smithi150.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-04-26T21:05:51.102 INFO:teuthology.orchestra.run.smithi171.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.8 MB/s | 4.7 MB 00:00 2024-04-26T21:05:51.111 INFO:teuthology.orchestra.run.smithi150.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 6.3 MB/s | 272 kB 00:00 2024-04-26T21:05:51.137 INFO:teuthology.orchestra.run.smithi150.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-26T21:05:51.162 INFO:teuthology.orchestra.run.smithi150.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-26T21:05:51.187 INFO:teuthology.orchestra.run.smithi150.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-04-26T21:05:51.213 INFO:teuthology.orchestra.run.smithi150.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-04-26T21:05:51.255 INFO:teuthology.orchestra.run.smithi150.stdout:(138/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.1 MB/s | 2.0 MB 00:00 2024-04-26T21:05:51.280 INFO:teuthology.orchestra.run.smithi150.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-04-26T21:05:51.305 INFO:teuthology.orchestra.run.smithi150.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-04-26T21:05:51.330 INFO:teuthology.orchestra.run.smithi150.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-26T21:05:51.356 INFO:teuthology.orchestra.run.smithi150.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-26T21:05:51.381 INFO:teuthology.orchestra.run.smithi150.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-26T21:05:51.406 INFO:teuthology.orchestra.run.smithi150.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-26T21:05:51.431 INFO:teuthology.orchestra.run.smithi150.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-26T21:05:51.457 INFO:teuthology.orchestra.run.smithi150.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-26T21:05:51.482 INFO:teuthology.orchestra.run.smithi150.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-26T21:05:51.599 INFO:teuthology.orchestra.run.smithi150.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-26T21:05:52.057 INFO:teuthology.orchestra.run.smithi150.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 2.9 MB/s | 1.3 MB 00:00 2024-04-26T21:05:52.069 INFO:teuthology.orchestra.run.smithi171.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 8.1 MB/s | 21 MB 00:02 2024-04-26T21:05:52.136 INFO:teuthology.orchestra.run.smithi171.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 379 kB/s | 25 kB 00:00 2024-04-26T21:05:52.320 INFO:teuthology.orchestra.run.smithi171.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 8.7 MB/s | 11 MB 00:01 2024-04-26T21:05:52.420 INFO:teuthology.orchestra.run.smithi171.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 225 kB 00:00 2024-04-26T21:05:52.483 INFO:teuthology.orchestra.run.smithi150.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 2.4 MB 00:00 2024-04-26T21:05:52.512 INFO:teuthology.orchestra.run.smithi171.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 295 kB/s | 27 kB 00:00 2024-04-26T21:05:52.595 INFO:teuthology.orchestra.run.smithi171.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 381 kB/s | 32 kB 00:00 2024-04-26T21:05:52.883 INFO:teuthology.orchestra.run.smithi150.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 2.4 MB/s | 3.4 MB 00:01 2024-04-26T21:05:52.904 INFO:teuthology.orchestra.run.smithi171.stdout:(14/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 7.8 MB/s | 17 MB 00:02 2024-04-26T21:05:52.929 INFO:teuthology.orchestra.run.smithi171.stdout:(15/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.2 MB/s | 747 kB 00:00 2024-04-26T21:05:52.955 INFO:teuthology.orchestra.run.smithi171.stdout:(16/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 3.2 MB/s | 163 kB 00:00 2024-04-26T21:05:52.980 INFO:teuthology.orchestra.run.smithi171.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 2.5 MB/s | 127 kB 00:00 2024-04-26T21:05:53.013 INFO:teuthology.orchestra.run.smithi171.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 8.4 MB/s | 501 kB 00:00 2024-04-26T21:05:53.089 INFO:teuthology.orchestra.run.smithi171.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 603 kB/s | 45 kB 00:00 2024-04-26T21:05:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-04-26T21:05:53.172 INFO:teuthology.orchestra.run.smithi171.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.6 MB/s | 135 kB 00:00 2024-04-26T21:05:53.246 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:05:53.256 INFO:teuthology.orchestra.run.smithi171.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.9 MB/s | 161 kB 00:00 2024-04-26T21:05:53.292 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:05:53.325 INFO:teuthology.orchestra.run.smithi150.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 1.7 MB/s | 3.1 MB 00:01 2024-04-26T21:05:53.328 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:05:53.328 INFO:teuthology.orchestra.run.smithi150.stdout:Total 12 MB/s | 222 MB 00:17 2024-04-26T21:05:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:05:53.348 INFO:teuthology.orchestra.run.smithi171.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.4 MB/s | 321 kB 00:00 2024-04-26T21:05:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:05:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:05:53.431 INFO:teuthology.orchestra.run.smithi171.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.5 MB/s | 298 kB 00:00 2024-04-26T21:05:53.448 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:05:53.515 INFO:teuthology.orchestra.run.smithi171.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T21:05:53.590 INFO:teuthology.orchestra.run.smithi171.stdout:(25/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 8.8 MB/s | 5.4 MB 00:00 2024-04-26T21:05:53.616 INFO:teuthology.orchestra.run.smithi171.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 853 kB/s | 85 kB 00:00 2024-04-26T21:05:53.700 INFO:teuthology.orchestra.run.smithi171.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-26T21:05:53.767 INFO:teuthology.orchestra.run.smithi171.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 514 kB/s | 34 kB 00:00 2024-04-26T21:05:53.775 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:05:53.851 INFO:teuthology.orchestra.run.smithi171.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.7 MB/s | 142 kB 00:00 2024-04-26T21:05:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:05:53.909 INFO:teuthology.orchestra.run.smithi171.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 9.7 MB/s | 3.1 MB 00:00 2024-04-26T21:05:54.003 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:05:54.041 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:05:54.132 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:05:54.162 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:05:54.309 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:05:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:05:54.376 INFO:teuthology.orchestra.run.smithi171.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 6.7 MB/s | 3.5 MB 00:00 2024-04-26T21:05:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:05:54.398 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-26T21:05:54.476 INFO:teuthology.orchestra.run.smithi171.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.4 MB/s | 249 kB 00:00 2024-04-26T21:05:54.482 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:05:54.504 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:05:54.543 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-26T21:05:54.543 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-26T21:05:54.550 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:05:54.560 INFO:teuthology.orchestra.run.smithi171.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 593 kB/s | 49 kB 00:00 2024-04-26T21:05:54.571 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:05:54.596 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:05:54.644 INFO:teuthology.orchestra.run.smithi171.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 197 kB/s | 16 kB 00:00 2024-04-26T21:05:54.744 INFO:teuthology.orchestra.run.smithi171.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 8.9 MB/s | 7.4 MB 00:00 2024-04-26T21:05:54.769 INFO:teuthology.orchestra.run.smithi171.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.1 MB/s | 273 kB 00:00 2024-04-26T21:05:54.789 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:05:54.821 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:05:54.853 INFO:teuthology.orchestra.run.smithi171.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 218 kB/s | 18 kB 00:00 2024-04-26T21:05:54.878 INFO:teuthology.orchestra.run.smithi171.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.6 MB/s | 762 kB 00:00 2024-04-26T21:05:54.928 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:05:54.937 INFO:teuthology.orchestra.run.smithi171.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 964 kB/s | 80 kB 00:00 2024-04-26T21:05:54.995 INFO:teuthology.orchestra.run.smithi171.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-26T21:05:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:05:55.037 INFO:teuthology.orchestra.run.smithi171.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-26T21:05:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:05:55.088 INFO:teuthology.orchestra.run.smithi171.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-26T21:05:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:05:55.154 INFO:teuthology.orchestra.run.smithi171.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T21:05:55.163 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:05:55.196 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:05:55.240 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:05:55.273 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:05:55.300 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:05:55.371 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:05:55.431 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:05:55.462 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:05:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:05:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:05:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:05:55.630 INFO:teuthology.orchestra.run.smithi171.stdout:(44/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 14 MB/s | 49 MB 00:03 2024-04-26T21:05:55.659 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:05:55.664 INFO:teuthology.orchestra.run.smithi171.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-04-26T21:05:55.688 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:05:55.705 INFO:teuthology.orchestra.run.smithi171.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 245 kB/s | 135 kB 00:00 2024-04-26T21:05:55.739 INFO:teuthology.orchestra.run.smithi171.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-26T21:05:55.749 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:05:55.764 INFO:teuthology.orchestra.run.smithi171.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-26T21:05:55.794 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:05:55.840 INFO:teuthology.orchestra.run.smithi171.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 5.0 MB/s | 689 kB 00:00 2024-04-26T21:05:55.865 INFO:teuthology.orchestra.run.smithi171.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-04-26T21:05:55.869 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:05:55.898 INFO:teuthology.orchestra.run.smithi171.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-04-26T21:05:55.924 INFO:teuthology.orchestra.run.smithi171.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 256 kB/s | 21 kB 00:00 2024-04-26T21:05:55.966 INFO:teuthology.orchestra.run.smithi171.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-04-26T21:05:55.999 INFO:teuthology.orchestra.run.smithi171.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-26T21:05:56.004 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:05:56.041 INFO:teuthology.orchestra.run.smithi171.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 241 kB/s | 18 kB 00:00 2024-04-26T21:05:56.054 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:05:56.058 INFO:teuthology.orchestra.run.smithi171.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-26T21:05:56.142 INFO:teuthology.orchestra.run.smithi171.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-26T21:05:56.167 INFO:teuthology.orchestra.run.smithi171.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-04-26T21:05:56.192 INFO:teuthology.orchestra.run.smithi171.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-26T21:05:56.242 INFO:teuthology.orchestra.run.smithi171.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 512 kB/s | 25 kB 00:00 2024-04-26T21:05:56.276 INFO:teuthology.orchestra.run.smithi171.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-04-26T21:05:56.302 INFO:teuthology.orchestra.run.smithi171.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 4.2 MB/s | 582 kB 00:00 2024-04-26T21:05:56.322 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-26T21:05:56.322 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-26T21:05:56.327 INFO:teuthology.orchestra.run.smithi171.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-04-26T21:05:56.377 INFO:teuthology.orchestra.run.smithi171.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-26T21:05:56.394 INFO:teuthology.orchestra.run.smithi171.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 528 kB/s | 35 kB 00:00 2024-04-26T21:05:56.419 INFO:teuthology.orchestra.run.smithi171.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-04-26T21:05:56.545 INFO:teuthology.orchestra.run.smithi171.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-26T21:05:56.574 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:05:56.637 INFO:teuthology.orchestra.run.smithi171.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-26T21:05:56.666 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:05:56.696 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:05:56.712 INFO:teuthology.orchestra.run.smithi171.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-04-26T21:05:56.737 INFO:teuthology.orchestra.run.smithi171.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-26T21:05:56.738 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:05:56.777 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:05:56.804 INFO:teuthology.orchestra.run.smithi171.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-04-26T21:05:56.818 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:05:56.862 INFO:teuthology.orchestra.run.smithi171.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T21:05:56.930 INFO:teuthology.orchestra.run.smithi171.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-04-26T21:05:56.987 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:05:57.055 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:05:57.056 INFO:teuthology.orchestra.run.smithi171.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-04-26T21:05:57.082 INFO:teuthology.orchestra.run.smithi171.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-04-26T21:05:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:05:57.132 INFO:teuthology.orchestra.run.smithi171.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-26T21:05:57.164 INFO:teuthology.orchestra.run.smithi171.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-26T21:05:57.199 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:05:57.214 INFO:teuthology.orchestra.run.smithi171.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-04-26T21:05:57.230 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:05:57.256 INFO:teuthology.orchestra.run.smithi171.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-26T21:05:57.267 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:05:57.281 INFO:teuthology.orchestra.run.smithi171.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-04-26T21:05:57.314 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:05:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:05:57.373 INFO:teuthology.orchestra.run.smithi171.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-26T21:05:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:05:57.424 INFO:teuthology.orchestra.run.smithi171.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-26T21:05:57.435 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:05:57.474 INFO:teuthology.orchestra.run.smithi171.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-26T21:05:57.499 INFO:teuthology.orchestra.run.smithi171.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-26T21:05:57.558 INFO:teuthology.orchestra.run.smithi171.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 504 kB/s | 42 kB 00:00 2024-04-26T21:05:57.570 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:05:57.600 INFO:teuthology.orchestra.run.smithi171.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-26T21:05:57.615 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:05:57.642 INFO:teuthology.orchestra.run.smithi171.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-26T21:05:57.645 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:05:57.687 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:05:57.692 INFO:teuthology.orchestra.run.smithi171.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-26T21:05:57.717 INFO:teuthology.orchestra.run.smithi171.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 680 kB/s | 51 kB 00:00 2024-04-26T21:05:57.779 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:05:57.784 INFO:teuthology.orchestra.run.smithi171.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.7 MB/s | 5.1 MB 00:00 2024-04-26T21:05:57.809 INFO:teuthology.orchestra.run.smithi171.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 458 kB/s | 54 kB 00:00 2024-04-26T21:05:57.820 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:05:57.860 INFO:teuthology.orchestra.run.smithi171.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-04-26T21:05:57.860 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:05:57.885 INFO:teuthology.orchestra.run.smithi171.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-04-26T21:05:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:05:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:05:57.935 INFO:teuthology.orchestra.run.smithi171.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-04-26T21:05:57.960 INFO:teuthology.orchestra.run.smithi171.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-26T21:05:57.984 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:05:58.011 INFO:teuthology.orchestra.run.smithi171.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-04-26T21:05:58.062 INFO:teuthology.orchestra.run.smithi171.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-26T21:05:58.086 INFO:teuthology.orchestra.run.smithi171.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-04-26T21:05:58.110 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:05:58.142 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:05:58.153 INFO:teuthology.orchestra.run.smithi171.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-04-26T21:05:58.170 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:05:58.179 INFO:teuthology.orchestra.run.smithi171.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 692 kB/s | 64 kB 00:00 2024-04-26T21:05:58.205 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:05:58.237 INFO:teuthology.orchestra.run.smithi171.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-04-26T21:05:58.252 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:05:58.284 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:05:58.339 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:05:58.373 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:05:58.388 INFO:teuthology.orchestra.run.smithi171.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-26T21:05:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:05:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:05:58.569 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:05:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:05:58.653 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:05:58.699 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:05:58.721 INFO:teuthology.orchestra.run.smithi171.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-26T21:05:58.797 INFO:teuthology.orchestra.run.smithi171.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-04-26T21:05:58.797 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:05:58.822 INFO:teuthology.orchestra.run.smithi171.stdout:(105/152): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-04-26T21:05:58.855 INFO:teuthology.orchestra.run.smithi171.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-04-26T21:05:58.905 INFO:teuthology.orchestra.run.smithi171.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-04-26T21:05:58.972 INFO:teuthology.orchestra.run.smithi171.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 729 kB/s | 49 kB 00:00 2024-04-26T21:05:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:05:59.023 INFO:teuthology.orchestra.run.smithi171.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-26T21:05:59.090 INFO:teuthology.orchestra.run.smithi171.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-26T21:05:59.190 INFO:teuthology.orchestra.run.smithi171.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-04-26T21:05:59.265 INFO:teuthology.orchestra.run.smithi171.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-26T21:05:59.365 INFO:teuthology.orchestra.run.smithi171.stdout:(113/152): libarrow-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-26T21:05:59.391 INFO:teuthology.orchestra.run.smithi171.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2024-04-26T21:05:59.416 INFO:teuthology.orchestra.run.smithi171.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 638 kB/s | 32 kB 00:00 2024-04-26T21:05:59.450 INFO:teuthology.orchestra.run.smithi171.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-26T21:05:59.475 INFO:teuthology.orchestra.run.smithi171.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-26T21:05:59.517 INFO:teuthology.orchestra.run.smithi171.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-26T21:05:59.626 INFO:teuthology.orchestra.run.smithi171.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-04-26T21:05:59.651 INFO:teuthology.orchestra.run.smithi171.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-04-26T21:05:59.713 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:05:59.718 INFO:teuthology.orchestra.run.smithi171.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-26T21:05:59.759 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:05:59.793 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:05:59.810 INFO:teuthology.orchestra.run.smithi171.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-26T21:05:59.869 INFO:teuthology.orchestra.run.smithi171.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-26T21:05:59.895 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:05:59.910 INFO:teuthology.orchestra.run.smithi171.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.9 MB/s | 2.0 MB 00:00 2024-04-26T21:05:59.936 INFO:teuthology.orchestra.run.smithi171.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-26T21:05:59.950 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:05:59.969 INFO:teuthology.orchestra.run.smithi171.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-04-26T21:05:59.995 INFO:teuthology.orchestra.run.smithi171.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-26T21:06:00.037 INFO:teuthology.orchestra.run.smithi171.stdout:(128/152): protobuf-compiler-3.14.0-13.el9.x86_ 480 kB/s | 863 kB 00:01 2024-04-26T21:06:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:06:00.062 INFO:teuthology.orchestra.run.smithi171.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-26T21:06:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:06:00.095 INFO:teuthology.orchestra.run.smithi171.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-26T21:06:00.120 INFO:teuthology.orchestra.run.smithi171.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-26T21:06:00.154 INFO:teuthology.orchestra.run.smithi171.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-04-26T21:06:00.180 INFO:teuthology.orchestra.run.smithi171.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-26T21:06:00.213 INFO:teuthology.orchestra.run.smithi171.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-26T21:06:00.238 INFO:teuthology.orchestra.run.smithi171.stdout:(135/152): python3-logutils-0.3.5-21.el9.noarch 230 kB/s | 46 kB 00:00 2024-04-26T21:06:00.264 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:06:00.264 INFO:teuthology.orchestra.run.smithi171.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-26T21:06:00.285 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:06:00.289 INFO:teuthology.orchestra.run.smithi171.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-04-26T21:06:00.314 INFO:teuthology.orchestra.run.smithi171.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-26T21:06:00.340 INFO:teuthology.orchestra.run.smithi171.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T21:06:00.362 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:06:00.365 INFO:teuthology.orchestra.run.smithi171.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-26T21:06:00.390 INFO:teuthology.orchestra.run.smithi171.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-04-26T21:06:00.415 INFO:teuthology.orchestra.run.smithi171.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-26T21:06:00.473 INFO:teuthology.orchestra.run.smithi171.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-26T21:06:00.500 INFO:teuthology.orchestra.run.smithi171.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 822 kB/s | 90 kB 00:00 2024-04-26T21:06:00.525 INFO:teuthology.orchestra.run.smithi171.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-26T21:06:00.550 INFO:teuthology.orchestra.run.smithi171.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 259 kB/s | 20 kB 00:00 2024-04-26T21:06:00.609 INFO:teuthology.orchestra.run.smithi171.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T21:06:00.651 INFO:teuthology.orchestra.run.smithi171.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-26T21:06:00.759 INFO:teuthology.orchestra.run.smithi171.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-26T21:06:00.893 INFO:teuthology.orchestra.run.smithi171.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-26T21:06:01.004 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:06:01.115 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:06:01.243 INFO:teuthology.orchestra.run.smithi171.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 4.9 MB/s | 3.4 MB 00:00 2024-04-26T21:06:01.318 INFO:teuthology.orchestra.run.smithi171.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-04-26T21:06:01.322 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:06:01.322 INFO:teuthology.orchestra.run.smithi171.stdout:Total 17 MB/s | 222 MB 00:12 2024-04-26T21:06:02.295 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-26T21:06:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:06:02.379 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:06:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:06:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:06:02.435 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-04-26T21:06:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:06:02.475 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:06:02.479 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:06:02.522 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:06:02.547 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:06:02.581 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-04-26T21:06:02.581 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-04-26T21:06:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:06:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:06:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:02.910 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:02.918 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:06:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:06:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:06:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:03.178 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:03.215 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:06:03.457 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:06:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:06:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:06:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:03.566 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:06:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:06:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:06:03.771 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:06:03.790 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:06:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:06:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:06:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:06:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:06:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:06:04.255 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:06:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:06:04.340 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-04-26T21:06:04.341 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-04-26T21:06:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:06:04.398 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:06:04.439 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:06:04.467 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:06:04.537 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:06:04.590 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:06:04.620 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:06:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:06:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:06:04.783 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:06:04.835 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:06:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:06:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:06:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:06:05.053 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:06:05.190 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:06:05.246 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:06:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:06:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:06:05.888 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:06:05.931 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:06:05.961 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:05.968 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:06:06.004 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:06.019 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:06:06.178 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:06:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:06:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:06:06.365 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:06:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:06:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:06:06.479 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:06:06.517 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:06:06.550 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:06.551 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:06:06.593 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:06:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:06.690 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:06:06.691 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:06:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:06:06.728 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:06:06.782 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:06:06.812 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:06:06.855 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:06:06.947 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:06:06.978 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:06:07.018 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:06:07.052 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:06:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:06:07.152 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:06:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:06:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:06:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:06:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:06:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:06:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:06:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:06:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:06:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:06:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:06:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:06:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:06:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:06:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:06:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:06:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:06:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:06:08.902 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:06:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:06:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:06:09.074 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:06:09.126 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:06:09.212 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:06:09.245 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:06:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:06:09.460 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:06:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:06:10.191 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:06:10.308 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:06:10.479 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-04-26T21:06:10.572 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:06:10.611 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:06:10.665 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:06:10.706 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:06:10.732 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:06:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:06:11.152 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:06:11.199 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:06:11.540 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:06:11.605 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:06:11.631 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:06:11.659 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:06:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:06:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:11.692 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:06:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:06:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:06:11.882 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:06:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:06:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:06:12.058 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:06:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:06:12.145 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:12.176 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:06:12.212 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:06:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:06:12.342 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:06:12.380 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:06:12.415 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:06:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:06:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:06:12.566 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:06:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:06:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:06:12.695 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:06:12.695 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:06:12.724 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:12.749 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:06:12.779 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:06:12.795 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:12.823 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:06:12.826 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:06:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:06:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:06:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:06:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:06:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:06:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:06:13.187 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:06:13.343 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:06:13.398 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:06:13.929 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:06:14.010 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:06:14.039 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:06:14.074 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:06:14.112 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:06:14.145 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:06:14.330 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:06:14.390 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:06:14.445 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:06:14.517 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:06:14.549 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:06:14.585 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:06:14.631 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:06:14.668 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:06:14.710 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:06:14.753 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:06:14.887 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:06:14.942 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:06:14.972 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:06:15.014 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:06:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:06:15.135 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:15.139 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:06:15.172 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:15.178 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:06:15.205 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:06:15.229 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:06:15.303 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:06:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:06:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:06:15.498 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:06:15.532 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:06:15.571 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:06:15.602 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:06:15.633 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:06:15.667 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:06:15.701 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:06:15.723 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:15.743 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:06:15.753 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:15.768 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:06:15.841 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:15.841 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:06:15.842 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:06:15.842 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:15.854 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:06:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:06:15.898 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:06:15.930 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:06:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:06:16.075 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:06:16.123 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:06:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:06:16.887 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:06:16.921 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:06:17.017 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:06:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:06:17.150 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:06:17.206 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:06:17.383 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:06:17.412 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:06:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:06:17.978 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:06:17.978 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-04-26T21:06:17.978 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-04-26T21:06:17.978 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-04-26T21:06:17.978 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-04-26T21:06:17.978 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-04-26T21:06:17.978 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-04-26T21:06:17.979 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-04-26T21:06:17.979 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:06:18.153 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:06:18.169 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:06:18.169 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:18.169 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:06:18.169 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:18.235 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:06:19.474 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:06:19.558 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:06:19.585 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:06:19.619 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:06:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:06:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:06:19.691 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:19.691 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:06:19.691 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:06:19.691 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:06:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:19.915 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:06:19.930 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:06:19.938 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:06:19.938 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:19.938 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:06:19.938 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:06:19.939 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:06:19.939 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:19.974 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:20.080 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:20.112 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:06:20.148 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:06:20.374 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:06:20.397 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:06:20.397 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:20.397 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:06:20.397 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:06:20.397 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:06:20.397 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:20.513 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:06:20.569 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:06:20.624 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:06:20.626 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:06:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:06:20.827 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:06:20.898 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:20.935 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:06:20.945 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:06:21.025 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:06:21.162 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:06:21.200 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:06:21.232 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:06:21.259 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:06:21.727 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:06:21.758 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:06:21.790 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:06:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:06:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:06:22.924 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:06:22.958 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:06:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:23.778 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:23.820 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:23.914 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:23.914 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:06:23.914 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:06:23.914 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:23.920 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:06:23.946 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:06:23.949 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:06:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:06:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:06:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:06:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:06:24.267 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:24.267 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:06:24.267 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:06:24.267 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:06:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:24.310 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:06:24.322 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:06:24.420 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:06:24.477 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:06:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:06:25.430 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:06:25.453 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:06:25.453 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:25.453 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:06:25.453 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:06:25.453 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:06:25.453 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:25.497 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:06:25.519 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:06:25.519 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:25.519 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:06:25.519 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:25.566 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:06:25.586 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:06:25.586 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:25.586 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:06:25.587 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:06:25.587 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:06:25.587 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:25.890 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:06:25.912 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:06:25.912 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:25.912 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:06:25.912 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:06:25.912 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:06:25.912 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:27.296 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:06:27.296 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-04-26T21:06:27.296 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-04-26T21:06:27.297 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-04-26T21:06:27.297 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-04-26T21:06:27.297 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-04-26T21:06:27.297 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-04-26T21:06:27.297 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-04-26T21:06:27.297 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:27.475 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:06:27.490 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:06:27.490 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:27.490 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:06:27.491 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:28.945 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:06:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:06:28.970 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:28.970 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:06:28.970 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:06:28.970 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:06:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:29.196 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:06:29.218 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:06:29.218 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:29.218 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:06:29.218 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:06:29.218 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:06:29.218 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:06:29.684 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:06:29.685 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:29.685 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:06:29.685 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:06:29.685 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:06:29.685 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:29.784 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:06:29.842 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:06:29.897 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:06:30.085 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:06:30.130 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:06:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:06:30.297 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:06:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:06:30.480 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:06:30.510 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:06:30.512 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:06:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:06:30.552 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:06:30.592 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:06:30.624 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:06:30.652 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:06:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:06:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:06:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:06:30.923 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:06:30.964 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:06:30.996 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:06:31.075 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:06:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:06:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:06:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:06:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:06:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:06:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:06:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:06:32.496 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:06:32.537 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:06:32.617 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:06:32.658 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:06:32.658 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:06:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:06:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:06:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:06:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:06:32.941 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:06:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:06:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:06:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:06:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:06:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:06:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:06:33.530 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:33.530 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:06:33.530 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:06:33.530 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:06:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:06:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:06:33.684 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:06:33.731 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:06:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:06:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:06:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:06:34.708 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:34.708 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:06:34.708 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:06:34.708 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:06:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:06:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:06:34.774 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:34.774 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:06:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:06:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:06:34.843 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:34.843 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:06:34.843 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:06:34.843 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:06:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:06:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:06:35.172 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:35.172 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:06:35.172 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:06:35.172 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:06:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:35.527 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:06:35.527 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-04-26T21:06:35.527 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-04-26T21:06:35.527 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-04-26T21:06:35.527 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-04-26T21:06:35.527 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-04-26T21:06:35.527 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-04-26T21:06:35.527 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-04-26T21:06:35.528 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:06:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:06:35.724 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:35.724 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:06:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:37.195 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:06:37.221 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:06:37.221 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:37.221 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:06:37.221 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:06:37.221 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:06:37.222 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:37.437 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:06:37.460 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:06:37.460 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:37.460 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:06:37.460 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:06:37.460 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:06:37.460 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:37.896 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:06:37.919 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:06:37.919 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:37.919 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:06:37.919 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:06:37.919 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:06:37.920 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:38.026 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:06:38.075 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:06:38.130 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:06:38.316 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:06:38.364 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:06:38.433 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:06:38.522 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:06:38.669 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:06:38.705 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:06:38.737 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:06:38.765 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:06:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:06:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:06:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:06:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:06:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:06:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:06:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:06:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:06:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:06:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:06:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:06:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:06:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:06:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:06:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:06:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:06:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:06:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:06:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:06:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:06:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:06:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:06:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:06:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:06:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:06:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:06:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:06:39.159 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:06:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:06:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:06:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:06:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:06:39.816 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:06:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:06:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:06:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:06:39.958 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:06:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:06:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:06:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:06:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:06:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:06:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:06:40.865 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:06:40.885 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:06:41.004 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:06:41.005 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.006 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:06:41.007 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:06:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:06:41.009 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:06:41.010 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:06:41.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:06:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:06:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:06:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:06:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:06:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:06:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:06:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:06:41.016 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-04-26T21:06:41.360 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:06:41.389 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:06:41.419 DEBUG:teuthology.parallel:result is None 2024-04-26T21:06:41.491 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:06:41.659 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:06:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:06:41.706 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:06:41.706 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:41.706 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:06:41.706 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:06:41.706 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:06:41.706 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:06:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:06:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:06:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:06:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:06:41.875 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:06:41.932 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:06:41.951 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:06:41.990 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:06:41.991 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:06:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:06:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:06:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:06:42.198 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:06:42.265 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:06:42.863 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:06:42.895 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:06:42.917 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:06:42.917 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:42.918 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:06:42.918 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:06:42.918 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:06:42.918 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:42.962 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:06:42.983 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:06:42.984 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:42.984 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:06:42.984 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:06:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:06:43.060 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:43.060 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:06:43.060 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:06:43.060 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:06:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:43.363 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:06:43.386 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:06:43.386 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:43.386 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:06:43.386 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:06:43.386 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:06:43.386 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:47.966 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:06:47.966 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:06:47.966 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:06:47.966 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:06:47.967 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:06:47.968 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:06:47.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:06:47.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:06:47.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:06:47.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:06:47.973 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:06:47.974 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:06:47.975 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:06:47.976 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:06:47.977 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:06:47.978 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:06:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:06:48.058 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:06:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:06:48.140 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:06:48.175 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:06:48.357 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:06:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:06:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:06:49.018 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:06:49.066 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:06:49.086 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:06:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:06:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:06:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:06:50.136 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:06:50.167 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:06:50.167 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:06:50.171 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:06:50.171 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:50.171 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.172 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.173 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:06:50.174 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:06:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:06:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:06:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:06:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:06:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:06:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.175 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:06:50.176 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:06:50.177 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:06:50.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:06:50.179 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:06:50.179 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:06:50.179 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:06:50.179 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:06:50.179 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:06:50.179 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:06:50.179 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:06:50.179 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:06:50.179 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:06:50.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:06:50.182 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:06:50.183 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:06:50.184 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:06:50.184 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:06:50.184 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:06:50.184 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:06:50.184 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-26T21:06:50.184 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-26T21:06:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:06:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:06:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:06:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:06:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:06:50.583 DEBUG:teuthology.parallel:result is None 2024-04-26T21:06:56.867 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:06:56.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:56.868 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:06:56.869 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:06:56.870 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:06:56.871 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:06:56.872 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:06:56.873 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:06:56.874 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:06:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:06:56.876 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:06:56.877 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:06:56.878 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:06:56.878 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:06:56.878 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:06:56.878 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:06:56.878 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:06:56.878 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:06:56.878 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:06:56.878 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:06:56.881 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:06:56.881 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:06:56.881 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:06:56.882 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:06:58.912 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:06:58.912 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:58.912 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-04-26T21:06:58.912 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.912 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.913 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:06:58.915 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:06:58.916 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:06:58.917 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:06:58.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:06:58.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:06:58.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:06:58.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.922 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:06:58.923 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:06:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:06:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:06:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:06:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:06:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:06:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:06:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:06:58.925 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:06:58.925 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:06:58.925 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-04-26T21:06:59.300 DEBUG:teuthology.parallel:result is None 2024-04-26T21:06:59.300 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:06:59.520 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:06:59.546 INFO:teuthology.orchestra.run.smithi129.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:06:59.546 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:06:59.546 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:06:59.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:06:59.762 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:06:59.788 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:06:59.788 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:06:59.789 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:06:59.791 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:07:00.006 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:07:00.027 INFO:teuthology.orchestra.run.smithi171.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:07:00.027 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:07:00.027 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:07:00.029 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:07:00.029 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:07:00.030 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:07:00.061 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:07:00.061 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:07:00.092 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:07:00.092 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:07:00.118 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:07:00.119 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:07:00.119 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:07:00.147 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:07:00.215 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:07:00.215 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:07:00.243 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:07:00.310 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:07:00.310 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:07:00.333 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:07:00.396 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:07:00.397 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:07:00.397 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:07:00.425 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:07:00.491 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:07:00.492 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:07:00.519 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:07:00.588 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:07:00.588 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:07:00.614 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:07:00.678 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:07:00.678 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:07:00.678 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:07:00.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:07:00.766 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:07:00.766 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:07:00.795 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:07:00.862 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:07:00.862 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:07:00.885 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:07:00.949 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T21:07:01.059 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2}} 2024-04-26T21:07:01.059 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:07:01.059 INFO:tasks.cephadm:Cluster fsid is eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:07:01.059 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T21:07:01.060 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.129', 'mon.b': '172.21.15.150', 'mon.c': '172.21.15.171'} 2024-04-26T21:07:01.060 INFO:tasks.cephadm:First mon is mon.a on smithi129 2024-04-26T21:07:01.060 INFO:tasks.cephadm:First mgr is x 2024-04-26T21:07:01.060 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T21:07:01.060 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname $(hostname -s) 2024-04-26T21:07:01.085 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-04-26T21:07:01.113 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-04-26T21:07:01.137 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T21:07:01.137 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:07:01.277 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T21:07:01.492 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:07:01.494 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-26T21:07:01.494 DEBUG:teuthology.orchestra.run.smithi129:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:07:02.068 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:07 /home/ubuntu/cephtest/cephadm 2024-04-26T21:07:02.068 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:07:02.557 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:07 /home/ubuntu/cephtest/cephadm 2024-04-26T21:07:02.557 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:07:03.033 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:07 /home/ubuntu/cephtest/cephadm 2024-04-26T21:07:03.034 DEBUG:teuthology.orchestra.run.smithi129:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:07:03.053 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:07:03.071 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:07:03.107 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-26T21:07:03.107 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:07:03.110 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:07:03.114 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:07:04.629 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:07:04.761 INFO:teuthology.orchestra.run.smithi129.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:07:04.787 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:07:32.711 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:07:32.711 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:07:32.712 INFO:teuthology.orchestra.run.smithi129.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:07:32.712 INFO:teuthology.orchestra.run.smithi129.stdout: "repo_digests": [ 2024-04-26T21:07:32.712 INFO:teuthology.orchestra.run.smithi129.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:07:32.712 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:07:32.712 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:07:33.846 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-26T21:07:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:07:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:07:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-04-26T21:07:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:07:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-04-26T21:07:33.847 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-26T21:07:41.723 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-26T21:07:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:07:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:07:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-04-26T21:07:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:07:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-04-26T21:07:41.723 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-26T21:07:41.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph 2024-04-26T21:07:41.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-04-26T21:07:41.813 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-04-26T21:07:41.840 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /etc/ceph 2024-04-26T21:07:41.868 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-04-26T21:07:41.896 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-04-26T21:07:41.918 INFO:tasks.cephadm:Writing seed config... 2024-04-26T21:07:41.919 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T21:07:41.920 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T21:07:41.921 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T21:07:41.922 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T21:07:41.923 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-26T21:07:41.926 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:07:41.926 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T21:07:41.942 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = eccca2b2-0410-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T21:07:41.943 DEBUG:teuthology.orchestra.run.smithi129:mon.a> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T21:07:41.985 DEBUG:teuthology.orchestra.run.smithi129:mgr.x> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T21:07:42.028 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T21:07:42.028 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid eccca2b2-0410-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.129 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:07:42.219 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:07:42.219 INFO:teuthology.orchestra.run.smithi129.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', 'eccca2b2-0410-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.129', '--skip-admin-label'] 2024-04-26T21:07:42.220 INFO:teuthology.orchestra.run.smithi129.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T21:07:42.220 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying podman|docker is present... 2024-04-26T21:07:42.257 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:07:42.257 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying lvm2 is present... 2024-04-26T21:07:42.258 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying time synchronization is in place... 2024-04-26T21:07:42.265 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:07:42.265 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:07:42.270 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:07:42.271 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-04-26T21:07:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout enabled 2024-04-26T21:07:42.283 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout active 2024-04-26T21:07:42.283 INFO:teuthology.orchestra.run.smithi129.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:07:42.283 INFO:teuthology.orchestra.run.smithi129.stdout:Repeating the final host check... 2024-04-26T21:07:42.306 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:07:42.307 INFO:teuthology.orchestra.run.smithi129.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T21:07:42.307 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl is present 2024-04-26T21:07:42.307 INFO:teuthology.orchestra.run.smithi129.stdout:lvcreate is present 2024-04-26T21:07:42.313 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:07:42.313 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:07:42.318 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:07:42.318 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-04-26T21:07:42.325 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout enabled 2024-04-26T21:07:42.331 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout active 2024-04-26T21:07:42.331 INFO:teuthology.orchestra.run.smithi129.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:07:42.331 INFO:teuthology.orchestra.run.smithi129.stdout:Host looks OK 2024-04-26T21:07:42.331 INFO:teuthology.orchestra.run.smithi129.stdout:Cluster fsid: eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:07:42.331 INFO:teuthology.orchestra.run.smithi129.stdout:Acquiring lock 139696053177504 on /run/cephadm/eccca2b2-0410-11ef-bc93-c7b262605968.lock 2024-04-26T21:07:42.331 INFO:teuthology.orchestra.run.smithi129.stdout:Lock 139696053177504 acquired on /run/cephadm/eccca2b2-0410-11ef-bc93-c7b262605968.lock 2024-04-26T21:07:42.331 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying IP 172.21.15.129 port 3300 ... 2024-04-26T21:07:42.332 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying IP 172.21.15.129 port 6789 ... 2024-04-26T21:07:42.332 INFO:teuthology.orchestra.run.smithi129.stdout:Base mon IP(s) is [172.21.15.129:3300, 172.21.15.129:6789], mon addrv is [v2:172.21.15.129:3300,v1:172.21.15.129:6789] 2024-04-26T21:07:42.335 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.129 metric 100 2024-04-26T21:07:42.335 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.129 metric 100 2024-04-26T21:07:42.337 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T21:07:42.337 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-26T21:07:42.338 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout inet6 fe80::71d5:251f:8718:9c5a/64 scope link noprefixroute 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:Mon IP `172.21.15.129` is in CIDR network `172.21.0.0/20` 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:Mon IP `172.21.15.129` is in CIDR network `172.21.0.0/20` 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T21:07:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:07:42.813 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:07:42.813 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:07:42.813 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T21:07:42.813 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-26T21:07:42.813 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T21:07:42.813 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:07:42.813 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T21:07:43.726 INFO:teuthology.orchestra.run.smithi129.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:07:43.726 INFO:teuthology.orchestra.run.smithi129.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:07:43.727 INFO:teuthology.orchestra.run.smithi129.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T21:07:44.875 INFO:teuthology.orchestra.run.smithi129.stdout:stat: stdout 167 167 2024-04-26T21:07:44.875 INFO:teuthology.orchestra.run.smithi129.stdout:Creating initial keys... 2024-04-26T21:07:45.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph-authtool: stdout AQChFyxm9ntxDBAA72UfYqsEcnEwF+F6j0MahA== 2024-04-26T21:07:46.676 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph-authtool: stdout AQCiFyxmYrXjBRAA2FIMLCBOBYaDmnDLPf6/Og== 2024-04-26T21:07:47.634 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph-authtool: stdout AQCiFyxmIE08OxAAOH8ESXctqxHfTNw0yZXg8w== 2024-04-26T21:07:47.635 INFO:teuthology.orchestra.run.smithi129.stdout:Creating initial monmap... 2024-04-26T21:07:48.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:07:48.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T21:07:48.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:07:48.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:07:48.476 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool for a [v2:172.21.15.129:3300,v1:172.21.15.129:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:07:48.476 INFO:teuthology.orchestra.run.smithi129.stdout:setting min_mon_release = quincy 2024-04-26T21:07:48.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: set fsid to eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:07:48.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:07:48.476 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:07:48.476 INFO:teuthology.orchestra.run.smithi129.stdout:Creating mon... 2024-04-26T21:07:54.561 INFO:teuthology.orchestra.run.smithi129.stdout:create mon.a on 2024-04-26T21:07:55.040 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T21:07:55.345 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T21:07:56.004 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-eccca2b2-0410-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-eccca2b2-0410-11ef-bc93-c7b262605968.target. 2024-04-26T21:07:56.005 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-eccca2b2-0410-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-eccca2b2-0410-11ef-bc93-c7b262605968.target. 2024-04-26T21:07:56.723 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.a 2024-04-26T21:07:56.723 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T21:07:56.971 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eccca2b2-0410-11ef-bc93-c7b262605968.target.wants/ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-eccca2b2-0410-11ef-bc93-c7b262605968@.service. 2024-04-26T21:07:57.766 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:07:57.766 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout disabled 2024-04-26T21:07:57.773 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:07:57.774 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-04-26T21:07:57.774 INFO:teuthology.orchestra.run.smithi129.stdout:firewalld.service is not enabled 2024-04-26T21:07:57.774 INFO:teuthology.orchestra.run.smithi129.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:07:57.774 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mon to start... 2024-04-26T21:07:57.774 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mon... 2024-04-26T21:07:57.899 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:57 smithi129 systemd[1]: Started Ceph mon.a for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:07:58.852 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:07:58.852 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: monmap epoch 1 2024-04-26T21:07:58.852 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:07:58.852 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: last_changed 2024-04-26T21:07:47.959085+0000 2024-04-26T21:07:58.852 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:07:58.852 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: min_mon_release 19 (squid) 2024-04-26T21:07:58.852 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: election_strategy: 1 2024-04-26T21:07:58.853 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:07:58.853 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: fsmap 2024-04-26T21:07:58.853 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:07:58.853 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: mgrmap e1: no daemons active 2024-04-26T21:07:58.853 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:58 smithi129 ceph-mon[28855]: from='client.? 172.21.15.129:0/5515064' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T21:07:58.861 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T21:07:58.861 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout id: eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:07:58.861 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T21:07:58.861 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:07:58.861 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout services: 2024-04-26T21:07:58.861 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.577737s) 2024-04-26T21:07:58.861 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout data: 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.220+0000 7fb9f9416700 1 Processor -- start 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.220+0000 7fb9f9416700 1 -- start start 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.220+0000 7fb9f9416700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f410bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.220+0000 7fb9f9416700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f407ce70 con 0x7fb9f4109770 2024-04-26T21:07:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.221+0000 7fb9f2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f410bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.221+0000 7fb9f2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f410bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39968/0 (socket says 172.21.15.129:39968) 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.221+0000 7fb9f2ffd700 1 -- 172.21.15.129:0/942999910 learned_addr learned my addr 172.21.15.129:0/942999910 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.221+0000 7fb9f2ffd700 1 -- 172.21.15.129:0/942999910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f410c0a0 con 0x7fb9f4109770 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.221+0000 7fb9f2ffd700 1 --2- 172.21.15.129:0/942999910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f410bb60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40098a0 tx=0x7fb9e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a741dc8f0f435 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.221+0000 7fb9f27fc700 1 -- 172.21.15.129:0/942999910 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9e4004160 con 0x7fb9f4109770 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.221+0000 7fb9f27fc700 1 -- 172.21.15.129:0/942999910 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e4004300 con 0x7fb9f4109770 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.221+0000 7fb9f27fc700 1 -- 172.21.15.129:0/942999910 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9e40036a0 con 0x7fb9f4109770 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.222+0000 7fb9f9416700 1 -- 172.21.15.129:0/942999910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 msgr2=0x7fb9f410bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.222+0000 7fb9f9416700 1 --2- 172.21.15.129:0/942999910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f410bb60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40098a0 tx=0x7fb9e4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.222+0000 7fb9f9416700 1 -- 172.21.15.129:0/942999910 shutdown_connections 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.222+0000 7fb9f9416700 1 --2- 172.21.15.129:0/942999910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f410bb60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.222+0000 7fb9f9416700 1 -- 172.21.15.129:0/942999910 >> 172.21.15.129:0/942999910 conn(0x7fb9f4103160 msgr2=0x7fb9f41055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:58.863 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.222+0000 7fb9f9416700 1 -- 172.21.15.129:0/942999910 shutdown_connections 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.222+0000 7fb9f9416700 1 -- 172.21.15.129:0/942999910 wait complete. 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.222+0000 7fb9f9416700 1 Processor -- start 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f9416700 1 -- start start 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f9416700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f9416700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f410c510 con 0x7fb9f4109770 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f419c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39978/0 (socket says 172.21.15.129:39978) 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f2ffd700 1 -- 172.21.15.129:0/5515064 learned_addr learned my addr 172.21.15.129:0/5515064 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f2ffd700 1 -- 172.21.15.129:0/5515064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f419c8e0 con 0x7fb9f4109770 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f2ffd700 1 --2- 172.21.15.129:0/5515064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f419c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4000c00 tx=0x7fb9e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f0ff9700 1 -- 172.21.15.129:0/5515064 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9e4003db0 con 0x7fb9f4109770 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f0ff9700 1 -- 172.21.15.129:0/5515064 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e4011630 con 0x7fb9f4109770 2024-04-26T21:07:58.864 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f0ff9700 1 -- 172.21.15.129:0/5515064 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9e4011950 con 0x7fb9f4109770 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f9416700 1 -- 172.21.15.129:0/5515064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f419cb70 con 0x7fb9f4109770 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.223+0000 7fb9f9416700 1 -- 172.21.15.129:0/5515064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f419f860 con 0x7fb9f4109770 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.224+0000 7fb9f0ff9700 1 -- 172.21.15.129:0/5515064 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb9e4011af0 con 0x7fb9f4109770 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.224+0000 7fb9f0ff9700 1 -- 172.21.15.129:0/5515064 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb9e401c450 con 0x7fb9f4109770 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.224+0000 7fb9f9416700 1 -- 172.21.15.129:0/5515064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f4064a80 con 0x7fb9f4109770 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.227+0000 7fb9f0ff9700 1 -- 172.21.15.129:0/5515064 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb9e4004260 con 0x7fb9f4109770 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.296+0000 7fb9f9416700 1 -- 172.21.15.129:0/5515064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb9f402e700 con 0x7fb9f4109770 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.296+0000 7fb9f0ff9700 1 -- 172.21.15.129:0/5515064 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb9e401b450 con 0x7fb9f4109770 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.297+0000 7fb9f9416700 1 -- 172.21.15.129:0/5515064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 msgr2=0x7fb9f419c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.297+0000 7fb9f9416700 1 --2- 172.21.15.129:0/5515064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f419c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4000c00 tx=0x7fb9e4004750 comp rx=0 tx=0).stop 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.297+0000 7fb9f9416700 1 -- 172.21.15.129:0/5515064 shutdown_connections 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.297+0000 7fb9f9416700 1 --2- 172.21.15.129:0/5515064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f4109770 0x7fb9f419c3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:58.865 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.297+0000 7fb9f9416700 1 -- 172.21.15.129:0/5515064 >> 172.21.15.129:0/5515064 conn(0x7fb9f4103160 msgr2=0x7fb9f41054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:58.866 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.297+0000 7fb9f9416700 1 -- 172.21.15.129:0/5515064 shutdown_connections 2024-04-26T21:07:58.866 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:58.297+0000 7fb9f9416700 1 -- 172.21.15.129:0/5515064 wait complete. 2024-04-26T21:07:58.866 INFO:teuthology.orchestra.run.smithi129.stdout:mon is available 2024-04-26T21:07:58.866 INFO:teuthology.orchestra.run.smithi129.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T21:08:00.083 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:59 smithi129 ceph-mon[28855]: from='client.? 172.21.15.129:0/3055817206' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:08:00.083 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:07:59 smithi129 ceph-mon[28855]: from='client.? 172.21.15.129:0/3055817206' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T21:08:00.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout fsid = eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.129:3300,v1:172.21.15.129:6789] 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:08:00.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.300+0000 7f892da77700 1 Processor -- start 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.300+0000 7f892da77700 1 -- start start 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.300+0000 7f892da77700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f892810bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.300+0000 7f892da77700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f892807ceb0 con 0x7f89281096f0 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.301+0000 7f89277fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f892810bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.301+0000 7f89277fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f892810bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39992/0 (socket says 172.21.15.129:39992) 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.301+0000 7f89277fe700 1 -- 172.21.15.129:0/889155057 learned_addr learned my addr 172.21.15.129:0/889155057 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.301+0000 7f89277fe700 1 -- 172.21.15.129:0/889155057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892810c020 con 0x7f89281096f0 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.301+0000 7f89277fe700 1 --2- 172.21.15.129:0/889155057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f892810bae0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8920009e80 tx=0x7f892000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0492a53e5d1fa4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.301+0000 7f89267fc700 1 -- 172.21.15.129:0/889155057 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8920017070 con 0x7f89281096f0 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.301+0000 7f89267fc700 1 -- 172.21.15.129:0/889155057 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8920003a20 con 0x7f89281096f0 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.301+0000 7f89267fc700 1 -- 172.21.15.129:0/889155057 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8920003d20 con 0x7f89281096f0 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.302+0000 7f892da77700 1 -- 172.21.15.129:0/889155057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 msgr2=0x7f892810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.302+0000 7f892da77700 1 --2- 172.21.15.129:0/889155057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f892810bae0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8920009e80 tx=0x7f892000b330 comp rx=0 tx=0).stop 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.302+0000 7f892da77700 1 -- 172.21.15.129:0/889155057 shutdown_connections 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.302+0000 7f892da77700 1 --2- 172.21.15.129:0/889155057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f892810bae0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.302+0000 7f892da77700 1 -- 172.21.15.129:0/889155057 >> 172.21.15.129:0/889155057 conn(0x7f8928103140 msgr2=0x7f8928105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.302+0000 7f892da77700 1 -- 172.21.15.129:0/889155057 shutdown_connections 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.302+0000 7f892da77700 1 -- 172.21.15.129:0/889155057 wait complete. 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.302+0000 7f892da77700 1 Processor -- start 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f892da77700 1 -- start start 2024-04-26T21:08:00.094 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f892da77700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f8928108f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f892da77700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f892810c4f0 con 0x7f89281096f0 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f89277fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f8928108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f89277fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f8928108f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39996/0 (socket says 172.21.15.129:39996) 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f89277fe700 1 -- 172.21.15.129:0/3055817206 learned_addr learned my addr 172.21.15.129:0/3055817206 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f89277fe700 1 -- 172.21.15.129:0/3055817206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89281094b0 con 0x7f89281096f0 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f89277fe700 1 --2- 172.21.15.129:0/3055817206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f8928108f70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f892807c2f0 tx=0x7f8920004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f8924ff9700 1 -- 172.21.15.129:0/3055817206 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8920022020 con 0x7f89281096f0 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f8924ff9700 1 -- 172.21.15.129:0/3055817206 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8920018040 con 0x7f89281096f0 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f892da77700 1 -- 172.21.15.129:0/3055817206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89281076a0 con 0x7f89281096f0 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f8924ff9700 1 -- 172.21.15.129:0/3055817206 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8920017070 con 0x7f89281096f0 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.303+0000 7f892da77700 1 -- 172.21.15.129:0/3055817206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8928107be0 con 0x7f89281096f0 2024-04-26T21:08:00.098 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.304+0000 7f8924ff9700 1 -- 172.21.15.129:0/3055817206 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8920025020 con 0x7f89281096f0 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.304+0000 7f8924ff9700 1 -- 172.21.15.129:0/3055817206 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f892000f050 con 0x7f89281096f0 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.304+0000 7f892da77700 1 -- 172.21.15.129:0/3055817206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8928064a80 con 0x7f89281096f0 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.307+0000 7f8924ff9700 1 -- 172.21.15.129:0/3055817206 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f892002c030 con 0x7f89281096f0 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.373+0000 7f892da77700 1 -- 172.21.15.129:0/3055817206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8928195d20 con 0x7f89281096f0 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.377+0000 7f8924ff9700 1 -- 172.21.15.129:0/3055817206 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+467 (secure 0 0 0) 0x7f892001e480 con 0x7f89281096f0 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.378+0000 7f8924ff9700 1 -- 172.21.15.129:0/3055817206 <== mon.0 v2:172.21.15.129:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8920035070 con 0x7f89281096f0 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.379+0000 7f892da77700 1 -- 172.21.15.129:0/3055817206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 msgr2=0x7f8928108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.379+0000 7f892da77700 1 --2- 172.21.15.129:0/3055817206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f8928108f70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f892807c2f0 tx=0x7f8920004ad0 comp rx=0 tx=0).stop 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.379+0000 7f892da77700 1 -- 172.21.15.129:0/3055817206 shutdown_connections 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.379+0000 7f892da77700 1 --2- 172.21.15.129:0/3055817206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89281096f0 0x7f8928108f70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.379+0000 7f892da77700 1 -- 172.21.15.129:0/3055817206 >> 172.21.15.129:0/3055817206 conn(0x7f8928103140 msgr2=0x7f8928103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.379+0000 7f892da77700 1 -- 172.21.15.129:0/3055817206 shutdown_connections 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:07:59.379+0000 7f892da77700 1 -- 172.21.15.129:0/3055817206 wait complete. 2024-04-26T21:08:00.099 INFO:teuthology.orchestra.run.smithi129.stdout:Generating new minimal ceph.conf... 2024-04-26T21:08:01.159 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:00 smithi129 ceph-mon[28855]: from='client.? 172.21.15.129:0/2054287214' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:01.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.476+0000 7f671e7d3700 1 Processor -- start 2024-04-26T21:08:01.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.476+0000 7f671e7d3700 1 -- start start 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f671e7d3700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f67181090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f671e7d3700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67181096c0 con 0x7f6718108cd0 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f6717fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f67181090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f6717fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f67181090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40000/0 (socket says 172.21.15.129:40000) 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f6717fff700 1 -- 172.21.15.129:0/1073698318 learned_addr learned my addr 172.21.15.129:0/1073698318 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f6717fff700 1 -- 172.21.15.129:0/1073698318 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6718109ed0 con 0x7f6718108cd0 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f6717fff700 1 --2- 172.21.15.129:0/1073698318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f67181090f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098a0 tx=0x7f670c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ab9c413e92c378 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f6716ffd700 1 -- 172.21.15.129:0/1073698318 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f670c004160 con 0x7f6718108cd0 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f6716ffd700 1 -- 172.21.15.129:0/1073698318 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f670c004300 con 0x7f6718108cd0 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.477+0000 7f6716ffd700 1 -- 172.21.15.129:0/1073698318 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f670c011af0 con 0x7f6718108cd0 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.478+0000 7f671e7d3700 1 -- 172.21.15.129:0/1073698318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 msgr2=0x7f67181090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.478+0000 7f671e7d3700 1 --2- 172.21.15.129:0/1073698318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f67181090f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098a0 tx=0x7f670c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:01.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.478+0000 7f671e7d3700 1 -- 172.21.15.129:0/1073698318 shutdown_connections 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.478+0000 7f671e7d3700 1 --2- 172.21.15.129:0/1073698318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f67181090f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.478+0000 7f671e7d3700 1 -- 172.21.15.129:0/1073698318 >> 172.21.15.129:0/1073698318 conn(0x7f67181042b0 msgr2=0x7f67181066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.478+0000 7f671e7d3700 1 -- 172.21.15.129:0/1073698318 shutdown_connections 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.478+0000 7f671e7d3700 1 -- 172.21.15.129:0/1073698318 wait complete. 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.478+0000 7f671e7d3700 1 Processor -- start 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f671e7d3700 1 -- start start 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f671e7d3700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f671819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f671e7d3700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f671810d280 con 0x7f6718108cd0 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f6717fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f671819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f6717fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f671819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40008/0 (socket says 172.21.15.129:40008) 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f6717fff700 1 -- 172.21.15.129:0/2054287214 learned_addr learned my addr 172.21.15.129:0/2054287214 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f6717fff700 1 -- 172.21.15.129:0/2054287214 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671819d1c0 con 0x7f6718108cd0 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f6717fff700 1 --2- 172.21.15.129:0/2054287214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f671819cc80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f670c009fb0 tx=0x7f670c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f67157fa700 1 -- 172.21.15.129:0/2054287214 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f670c0036a0 con 0x7f6718108cd0 2024-04-26T21:08:01.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.479+0000 7f67157fa700 1 -- 172.21.15.129:0/2054287214 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f670c01a430 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.480+0000 7f671e7d3700 1 -- 172.21.15.129:0/2054287214 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671819d450 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.480+0000 7f67157fa700 1 -- 172.21.15.129:0/2054287214 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f670c019550 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.480+0000 7f671e7d3700 1 -- 172.21.15.129:0/2054287214 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67181a0140 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.480+0000 7f67157fa700 1 -- 172.21.15.129:0/2054287214 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f670c019be0 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.480+0000 7f67157fa700 1 -- 172.21.15.129:0/2054287214 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f670c0248b0 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.481+0000 7f671e7d3700 1 -- 172.21.15.129:0/2054287214 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6718064a80 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.483+0000 7f67157fa700 1 -- 172.21.15.129:0/2054287214 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f670c043ec0 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.553+0000 7f671e7d3700 1 -- 172.21.15.129:0/2054287214 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f67181a0310 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.553+0000 7f67157fa700 1 -- 172.21.15.129:0/2054287214 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f670c032020 con 0x7f6718108cd0 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.554+0000 7f671e7d3700 1 -- 172.21.15.129:0/2054287214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 msgr2=0x7f671819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.554+0000 7f671e7d3700 1 --2- 172.21.15.129:0/2054287214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f671819cc80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f670c009fb0 tx=0x7f670c011940 comp rx=0 tx=0).stop 2024-04-26T21:08:01.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.554+0000 7f671e7d3700 1 -- 172.21.15.129:0/2054287214 shutdown_connections 2024-04-26T21:08:01.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.554+0000 7f671e7d3700 1 --2- 172.21.15.129:0/2054287214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6718108cd0 0x7f671819cc80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:01.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.554+0000 7f671e7d3700 1 -- 172.21.15.129:0/2054287214 >> 172.21.15.129:0/2054287214 conn(0x7f67181042b0 msgr2=0x7f6718104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:01.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.554+0000 7f671e7d3700 1 -- 172.21.15.129:0/2054287214 shutdown_connections 2024-04-26T21:08:01.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:00.554+0000 7f671e7d3700 1 -- 172.21.15.129:0/2054287214 wait complete. 2024-04-26T21:08:01.170 INFO:teuthology.orchestra.run.smithi129.stdout:Restarting the monitor... 2024-04-26T21:08:01.430 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:01 smithi129 systemd[1]: Stopping Ceph mon.a for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:08:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:01 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a[28851]: 2024-04-26T21:08:01.429+0000 7fd18bcf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:08:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:01 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a[28851]: 2024-04-26T21:08:01.429+0000 7fd18bcf4700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T21:08:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:01 smithi129 podman[29102]: 2024-04-26 21:08:01.603310528 +0000 UTC m=+0.276472860 container died 71aa52c198470b860fd225037929b2af808897de0b36e3e805065cd7e39a63ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:08:02.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:01 smithi129 podman[29102]: 2024-04-26 21:08:01.971203953 +0000 UTC m=+0.644366284 container cleanup 71aa52c198470b860fd225037929b2af808897de0b36e3e805065cd7e39a63ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:08:02.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:01 smithi129 bash[29102]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a 2024-04-26T21:08:02.870 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:02 smithi129 podman[29116]: 2024-04-26 21:08:02.472465445 +0000 UTC m=+0.866852079 container remove 71aa52c198470b860fd225037929b2af808897de0b36e3e805065cd7e39a63ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T21:08:02.871 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:02 smithi129 systemd[1]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T21:08:02.871 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:02 smithi129 systemd[1]: Stopped Ceph mon.a for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:08:02.871 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:02 smithi129 systemd[1]: Starting Ceph mon.a for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:08:03.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:02 smithi129 podman[29207]: 2024-04-26 21:08:02.773197542 +0000 UTC m=+0.019000651 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:08:03.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:03 smithi129 podman[29207]: 2024-04-26 21:08:03.683475432 +0000 UTC m=+0.929278532 container create 7a96e43a75754ab6173a4b3092c569ac880dbeb52ee14c49f898a5c2d2b287fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:08:05.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 podman[29207]: 2024-04-26 21:08:05.534305449 +0000 UTC m=+2.780108539 container init 7a96e43a75754ab6173a4b3092c569ac880dbeb52ee14c49f898a5c2d2b287fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:08:05.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 podman[29207]: 2024-04-26 21:08:05.538285706 +0000 UTC m=+2.784088797 container start 7a96e43a75754ab6173a4b3092c569ac880dbeb52ee14c49f898a5c2d2b287fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: pidfile_write: ignore empty --pid-file 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: load: jerasure load: lrc 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Git sha 0 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: DB SUMMARY 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: DB Session ID: 98FJWUS5W363ZZUN9VXV 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T21:08:05.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104080 ; 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.env: 0x55865abb8cc0 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.info_log: 0x55865c625d80 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.statistics: (nil) 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.use_fsync: 0 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:08:05.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.db_log_dir: 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.wal_dir: 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:08:05.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.write_buffer_manager: 0x55865c5f70e0 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:08:05.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.unordered_write: 0 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.row_cache: None 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.wal_filter: None 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.wal_compression: 0 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:08:05.976 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:08:05.977 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_open_files: -1 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:08:05.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Compression algorithms supported: 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: kZSTD supported: 0 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T21:08:05.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.merge_operator: 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_filter: None 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55865c625f40) 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:08:05.980 INFO:journalctl@ceph.mon.a.smithi129.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: index_type: 0 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: data_block_index_type: 0 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: index_shortening: 1 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: checksum: 4 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: no_block_cache: 0 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_cache: 0x55865c6291f0 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_cache_options: 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: capacity : 536870912 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: num_shard_bits : 4 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: strict_capacity_limit : 0 2024-04-26T21:08:05.981 INFO:journalctl@ceph.mon.a.smithi129.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_cache_compressed: (nil) 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: persistent_cache: (nil) 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_size: 4096 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_size_deviation: 10 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_restart_interval: 16 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: index_block_restart_interval: 1 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: metadata_block_size: 4096 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: partition_filters: 0 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: use_delta_encoding: 1 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: filter_policy: bloomfilter 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: whole_key_filtering: 1 2024-04-26T21:08:05.982 INFO:journalctl@ceph.mon.a.smithi129.stdout: verify_compression: 0 2024-04-26T21:08:05.983 INFO:journalctl@ceph.mon.a.smithi129.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:08:05.983 INFO:journalctl@ceph.mon.a.smithi129.stdout: format_version: 5 2024-04-26T21:08:05.983 INFO:journalctl@ceph.mon.a.smithi129.stdout: enable_index_compression: 1 2024-04-26T21:08:05.983 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_align: 0 2024-04-26T21:08:05.983 INFO:journalctl@ceph.mon.a.smithi129.stdout: max_auto_readahead_size: 262144 2024-04-26T21:08:05.983 INFO:journalctl@ceph.mon.a.smithi129.stdout: prepopulate_block_cache: 0 2024-04-26T21:08:05.983 INFO:journalctl@ceph.mon.a.smithi129.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:08:05.983 INFO:journalctl@ceph.mon.a.smithi129.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:08:05.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:08:05.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:08:05.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression: NoCompression 2024-04-26T21:08:05.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:08:05.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:08:05.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:08:05.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.num_levels: 7 2024-04-26T21:08:05.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:08:05.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:08:05.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:08:05.988 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.ttl: 2592000 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:08:05.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ea2bc84b-2cfb-43e9-b839-cd33fcb15743 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165685570440, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T21:08:05.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165685571560, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99308, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97254, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13917, "raw_average_key_size": 49, "raw_value_size": 89648, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714165685, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ea2bc84b-2cfb-43e9-b839-cd33fcb15743", "db_session_id": "98FJWUS5W363ZZUN9VXV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165685571655, "job": 1, "event": "recovery_finished"} 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55865c746000 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: DB pointer 0x55865c732000 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout: ** DB Stats ** 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:08:05.991 INFO:journalctl@ceph.mon.a.smithi129.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:08:05.992 INFO:journalctl@ceph.mon.a.smithi129.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:08:05.992 INFO:journalctl@ceph.mon.a.smithi129.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:08:05.992 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-04-26T21:08:05.992 INFO:journalctl@ceph.mon.a.smithi129.stdout: ** Compaction Stats [default] ** 2024-04-26T21:08:05.992 INFO:journalctl@ceph.mon.a.smithi129.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:08:05.992 INFO:journalctl@ceph.mon.a.smithi129.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: L0 2/0 98.84 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: Sum 2/0 98.84 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: ** Compaction Stats [default] ** 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 129.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:08:05.993 INFO:journalctl@ceph.mon.a.smithi129.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: Cumulative compaction: 0.00 GB write, 28.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: Interval compaction: 0.00 GB write, 28.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: Block cache BinnedLRUCache@0x55865c6291f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: starting mon.a rank 0 at public addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] at bind addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mon.a@-1(???) e1 preinit fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mon.a@-1(???).mds e1 new map 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mon.a@-1(???).mds e1 print_map 2024-04-26T21:08:05.994 INFO:journalctl@ceph.mon.a.smithi129.stdout: e1 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout: legacy client fscid: -1 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout: No filesystems configured 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: monmap epoch 1 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:08:05.995 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: last_changed 2024-04-26T21:07:47.959085+0000 2024-04-26T21:08:05.996 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:08:05.996 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: min_mon_release 19 (squid) 2024-04-26T21:08:05.996 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: election_strategy: 1 2024-04-26T21:08:05.996 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:08:05.996 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: fsmap 2024-04-26T21:08:05.996 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:08:05.996 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:05 smithi129 ceph-mon[29237]: mgrmap e1: no daemons active 2024-04-26T21:08:06.323 INFO:teuthology.orchestra.run.smithi129.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T21:08:06.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:06 smithi129 bash[29207]: 7a96e43a75754ab6173a4b3092c569ac880dbeb52ee14c49f898a5c2d2b287fa 2024-04-26T21:08:06.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:06 smithi129 systemd[1]: Started Ceph mon.a for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:08:07.753 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.187+0000 7f5513e88700 1 Processor -- start 2024-04-26T21:08:07.753 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.187+0000 7f5513e88700 1 -- start start 2024-04-26T21:08:07.753 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.187+0000 7f5513e88700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c1079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:07.753 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.187+0000 7f5513e88700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f550c07ce70 con 0x7f550c107580 2024-04-26T21:08:07.753 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.188+0000 7f5511c24700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c1079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.188+0000 7f5511c24700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c1079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37860/0 (socket says 172.21.15.129:37860) 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.188+0000 7f5511c24700 1 -- 172.21.15.129:0/1680088205 learned_addr learned my addr 172.21.15.129:0/1680088205 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.188+0000 7f5511c24700 1 -- 172.21.15.129:0/1680088205 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c107ee0 con 0x7f550c107580 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.188+0000 7f5511c24700 1 --2- 172.21.15.129:0/1680088205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c1079a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55080098a0 tx=0x7f5508009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d1e845ae8bbce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.188+0000 7f5510c22700 1 -- 172.21.15.129:0/1680088205 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5508004160 con 0x7f550c107580 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.188+0000 7f5510c22700 1 -- 172.21.15.129:0/1680088205 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5508004300 con 0x7f550c107580 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.188+0000 7f5510c22700 1 -- 172.21.15.129:0/1680088205 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5508011af0 con 0x7f550c107580 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.189+0000 7f5513e88700 1 -- 172.21.15.129:0/1680088205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 msgr2=0x7f550c1079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.189+0000 7f5513e88700 1 --2- 172.21.15.129:0/1680088205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c1079a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55080098a0 tx=0x7f5508009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.189+0000 7f5513e88700 1 -- 172.21.15.129:0/1680088205 shutdown_connections 2024-04-26T21:08:07.754 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.189+0000 7f5513e88700 1 --2- 172.21.15.129:0/1680088205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c1079a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.189+0000 7f5513e88700 1 -- 172.21.15.129:0/1680088205 >> 172.21.15.129:0/1680088205 conn(0x7f550c1031c0 msgr2=0x7f550c1055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.189+0000 7f5513e88700 1 -- 172.21.15.129:0/1680088205 shutdown_connections 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.189+0000 7f5513e88700 1 -- 172.21.15.129:0/1680088205 wait complete. 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.190+0000 7f5513e88700 1 Processor -- start 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.190+0000 7f5513e88700 1 -- start start 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.190+0000 7f5513e88700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.190+0000 7f5513e88700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f550c108f10 con 0x7f550c107580 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.190+0000 7f5511c24700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.190+0000 7f5511c24700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37870/0 (socket says 172.21.15.129:37870) 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.190+0000 7f5511c24700 1 -- 172.21.15.129:0/4126321337 learned_addr learned my addr 172.21.15.129:0/4126321337 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.190+0000 7f5511c24700 1 -- 172.21.15.129:0/4126321337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c19f2e0 con 0x7f550c107580 2024-04-26T21:08:07.755 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.190+0000 7f5511c24700 1 --2- 172.21.15.129:0/4126321337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c19eda0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5508003f40 tx=0x7f5508011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.191+0000 7f5502ffd700 1 -- 172.21.15.129:0/4126321337 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55080038b0 con 0x7f550c107580 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.191+0000 7f5502ffd700 1 -- 172.21.15.129:0/4126321337 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5508003a50 con 0x7f550c107580 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.191+0000 7f5513e88700 1 -- 172.21.15.129:0/4126321337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550c19f570 con 0x7f550c107580 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.191+0000 7f5502ffd700 1 -- 172.21.15.129:0/4126321337 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f550801a5c0 con 0x7f550c107580 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.191+0000 7f5513e88700 1 -- 172.21.15.129:0/4126321337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550c1a2260 con 0x7f550c107580 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.191+0000 7f5502ffd700 1 -- 172.21.15.129:0/4126321337 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f550801ac50 con 0x7f550c107580 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.191+0000 7f5502ffd700 1 -- 172.21.15.129:0/4126321337 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5508019700 con 0x7f550c107580 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.192+0000 7f5513e88700 1 -- 172.21.15.129:0/4126321337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f550c064a80 con 0x7f550c107580 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.194+0000 7f5502ffd700 1 -- 172.21.15.129:0/4126321337 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f5508019900 con 0x7f550c107580 2024-04-26T21:08:07.756 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.262+0000 7f5513e88700 1 -- 172.21.15.129:0/4126321337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f550c02e710 con 0x7f550c107580 2024-04-26T21:08:07.757 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.263+0000 7f5502ffd700 1 -- 172.21.15.129:0/4126321337 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f5508019ae0 con 0x7f550c107580 2024-04-26T21:08:07.757 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.264+0000 7f5513e88700 1 -- 172.21.15.129:0/4126321337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 msgr2=0x7f550c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:07.757 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.264+0000 7f5513e88700 1 --2- 172.21.15.129:0/4126321337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c19eda0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5508003f40 tx=0x7f5508011890 comp rx=0 tx=0).stop 2024-04-26T21:08:07.757 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.264+0000 7f5513e88700 1 -- 172.21.15.129:0/4126321337 shutdown_connections 2024-04-26T21:08:07.757 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.264+0000 7f5513e88700 1 --2- 172.21.15.129:0/4126321337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f550c107580 0x7f550c19eda0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:07.757 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.264+0000 7f5513e88700 1 -- 172.21.15.129:0/4126321337 >> 172.21.15.129:0/4126321337 conn(0x7f550c1031c0 msgr2=0x7f550c195a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:07.757 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.264+0000 7f5513e88700 1 -- 172.21.15.129:0/4126321337 shutdown_connections 2024-04-26T21:08:07.757 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:07.264+0000 7f5513e88700 1 -- 172.21.15.129:0/4126321337 wait complete. 2024-04-26T21:08:07.757 INFO:teuthology.orchestra.run.smithi129.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T21:08:07.778 INFO:teuthology.orchestra.run.smithi129.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:08:07.778 INFO:teuthology.orchestra.run.smithi129.stdout:Creating mgr... 2024-04-26T21:08:07.778 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T21:08:07.779 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T21:08:08.038 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.x 2024-04-26T21:08:08.038 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T21:08:08.181 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eccca2b2-0410-11ef-bc93-c7b262605968.target.wants/ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-eccca2b2-0410-11ef-bc93-c7b262605968@.service. 2024-04-26T21:08:08.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:08 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/4126321337' entity='client.admin' 2024-04-26T21:08:08.470 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:08 smithi129 podman[29449]: 2024-04-26 21:08:08.421975879 +0000 UTC m=+0.119470381 container create 74221cb1dd2d5e10d23d3e4eb297ae834f20a26d609de66397fbdf148b443d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T21:08:08.715 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:08:08.715 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout disabled 2024-04-26T21:08:08.723 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:08:08.723 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-04-26T21:08:08.723 INFO:teuthology.orchestra.run.smithi129.stdout:firewalld.service is not enabled 2024-04-26T21:08:08.723 INFO:teuthology.orchestra.run.smithi129.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:08:08.731 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:08:08.731 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout disabled 2024-04-26T21:08:08.739 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:08:08.739 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-04-26T21:08:08.739 INFO:teuthology.orchestra.run.smithi129.stdout:firewalld.service is not enabled 2024-04-26T21:08:08.739 INFO:teuthology.orchestra.run.smithi129.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T21:08:08.739 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mgr to start... 2024-04-26T21:08:08.739 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mgr... 2024-04-26T21:08:08.778 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:08 smithi129 podman[29449]: 2024-04-26 21:08:08.616269679 +0000 UTC m=+0.313764180 container init 74221cb1dd2d5e10d23d3e4eb297ae834f20a26d609de66397fbdf148b443d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T21:08:08.778 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:08 smithi129 podman[29449]: 2024-04-26 21:08:08.618792177 +0000 UTC m=+0.316286686 container start 74221cb1dd2d5e10d23d3e4eb297ae834f20a26d609de66397fbdf148b443d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T21:08:08.779 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:08 smithi129 bash[29449]: 74221cb1dd2d5e10d23d3e4eb297ae834f20a26d609de66397fbdf148b443d5c 2024-04-26T21:08:08.779 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:08 smithi129 systemd[1]: Started Ceph mgr.x for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:08:09.045 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:08 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:08.777+0000 7f75ffd02200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:08:09.045 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:08 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:08.965+0000 7f75ffd02200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:09.394 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:09 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1793266152' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:08:09.394 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:09.069+0000 7f75ffd02200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:08:09.394 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:09.393+0000 7f75ffd02200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:08:09.720 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:09.479+0000 7f75ffd02200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:08:09.778 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsid": "eccca2b2-0410-11ef-bc93-c7b262605968", 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:09.779 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:08:09.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:08:09.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:07:57.721034+0000", 2024-04-26T21:08:09.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.145+0000 7f5bc40f3700 1 Processor -- start 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.145+0000 7f5bc40f3700 1 -- start start 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.145+0000 7f5bc40f3700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.145+0000 7f5bc40f3700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc108570 con 0x7f5bbc107b80 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.146+0000 7f5bc1e8f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.146+0000 7f5bc1e8f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37904/0 (socket says 172.21.15.129:37904) 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.146+0000 7f5bc1e8f700 1 -- 172.21.15.129:0/2605221570 learned_addr learned my addr 172.21.15.129:0/2605221570 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.146+0000 7f5bc1e8f700 1 -- 172.21.15.129:0/2605221570 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc108d80 con 0x7f5bbc107b80 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.146+0000 7f5bc1e8f700 1 --2- 172.21.15.129:0/2605221570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc107fa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00bb40 tx=0x7f5bac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702b4fa396476b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.146+0000 7f5bc0e8d700 1 -- 172.21.15.129:0/2605221570 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bac00dbe0 con 0x7f5bbc107b80 2024-04-26T21:08:09.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.146+0000 7f5bc0e8d700 1 -- 172.21.15.129:0/2605221570 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5bac0044c0 con 0x7f5bbc107b80 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.146+0000 7f5bc0e8d700 1 -- 172.21.15.129:0/2605221570 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bac010470 con 0x7f5bbc107b80 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.147+0000 7f5bc40f3700 1 -- 172.21.15.129:0/2605221570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 msgr2=0x7f5bbc107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.147+0000 7f5bc40f3700 1 --2- 172.21.15.129:0/2605221570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc107fa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00bb40 tx=0x7f5bac00be50 comp rx=0 tx=0).stop 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.147+0000 7f5bc40f3700 1 -- 172.21.15.129:0/2605221570 shutdown_connections 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.147+0000 7f5bc40f3700 1 --2- 172.21.15.129:0/2605221570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc107fa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.147+0000 7f5bc40f3700 1 -- 172.21.15.129:0/2605221570 >> 172.21.15.129:0/2605221570 conn(0x7f5bbc103140 msgr2=0x7f5bbc105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.147+0000 7f5bc40f3700 1 -- 172.21.15.129:0/2605221570 shutdown_connections 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.147+0000 7f5bc40f3700 1 -- 172.21.15.129:0/2605221570 wait complete. 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.148+0000 7f5bc40f3700 1 Processor -- start 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.148+0000 7f5bc40f3700 1 -- start start 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.148+0000 7f5bc40f3700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.148+0000 7f5bc40f3700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc07bfd0 con 0x7f5bbc107b80 2024-04-26T21:08:09.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.148+0000 7f5bc1e8f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.148+0000 7f5bc1e8f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37920/0 (socket says 172.21.15.129:37920) 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.148+0000 7f5bc1e8f700 1 -- 172.21.15.129:0/1793266152 learned_addr learned my addr 172.21.15.129:0/1793266152 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.148+0000 7f5bc1e8f700 1 -- 172.21.15.129:0/1793266152 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc19af20 con 0x7f5bbc107b80 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.149+0000 7f5bc1e8f700 1 --2- 172.21.15.129:0/1793266152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc19a9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00bfd0 tx=0x7f5bac003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.149+0000 7f5bb2ffd700 1 -- 172.21.15.129:0/1793266152 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bac027070 con 0x7f5bbc107b80 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.149+0000 7f5bb2ffd700 1 -- 172.21.15.129:0/1793266152 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5bac010c00 con 0x7f5bbc107b80 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.149+0000 7f5bc40f3700 1 -- 172.21.15.129:0/1793266152 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc19b1b0 con 0x7f5bbc107b80 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.149+0000 7f5bb2ffd700 1 -- 172.21.15.129:0/1793266152 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bac022c10 con 0x7f5bbc107b80 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.149+0000 7f5bc40f3700 1 -- 172.21.15.129:0/1793266152 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc19de50 con 0x7f5bbc107b80 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.149+0000 7f5bb2ffd700 1 -- 172.21.15.129:0/1793266152 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5bac02a070 con 0x7f5bbc107b80 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.149+0000 7f5bb2ffd700 1 -- 172.21.15.129:0/1793266152 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5bac01f070 con 0x7f5bbc107b80 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.150+0000 7f5bc40f3700 1 -- 172.21.15.129:0/1793266152 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc064a80 con 0x7f5bbc107b80 2024-04-26T21:08:09.785 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.152+0000 7f5bb2ffd700 1 -- 172.21.15.129:0/1793266152 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f5bac031030 con 0x7f5bbc107b80 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.224+0000 7f5bc40f3700 1 -- 172.21.15.129:0/1793266152 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5bbc193f40 con 0x7f5bbc107b80 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.224+0000 7f5bb2ffd700 1 -- 172.21.15.129:0/1793266152 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5bac015070 con 0x7f5bbc107b80 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.225+0000 7f5bc40f3700 1 -- 172.21.15.129:0/1793266152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 msgr2=0x7f5bbc19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.225+0000 7f5bc40f3700 1 --2- 172.21.15.129:0/1793266152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc19a9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00bfd0 tx=0x7f5bac003ff0 comp rx=0 tx=0).stop 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.225+0000 7f5bc40f3700 1 -- 172.21.15.129:0/1793266152 shutdown_connections 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.225+0000 7f5bc40f3700 1 --2- 172.21.15.129:0/1793266152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bbc107b80 0x7f5bbc19a9e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.225+0000 7f5bc40f3700 1 -- 172.21.15.129:0/1793266152 >> 172.21.15.129:0/1793266152 conn(0x7f5bbc103140 msgr2=0x7f5bbc191000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.226+0000 7f5bc40f3700 1 -- 172.21.15.129:0/1793266152 shutdown_connections 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:09.226+0000 7f5bc40f3700 1 -- 172.21.15.129:0/1793266152 wait complete. 2024-04-26T21:08:09.786 INFO:teuthology.orchestra.run.smithi129.stdout:mgr not available, waiting (1/15)... 2024-04-26T21:08:10.970 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:10 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:10.669+0000 7f75ffd02200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:08:10.970 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:10 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:10.834+0000 7f75ffd02200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:11.284 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:11 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:11.004+0000 7f75ffd02200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:08:11.284 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:11 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:11.283+0000 7f75ffd02200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:08:11.579 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:11 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:11.374+0000 7f75ffd02200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:08:11.579 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:11 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:11.577+0000 7f75ffd02200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:08:11.900 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:11 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:11.766+0000 7f75ffd02200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:12.693 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:12 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:12.371+0000 7f75ffd02200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:08:12.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:12 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1003050544' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:08:12.970 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:12 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:12.792+0000 7f75ffd02200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:08:12.970 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:12 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:12.891+0000 7f75ffd02200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:08:13.345 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:13 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:13.093+0000 7f75ffd02200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:08:13.494 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:13.494 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:08:13.494 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsid": "eccca2b2-0410-11ef-bc93-c7b262605968", 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T21:08:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:08:13.496 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:13.496 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:08:13.496 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:08:13.496 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:13.496 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:08:13.496 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:13.496 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:08:13.496 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:08:13.496 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:08:13.504 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:08:13.504 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:08:13.504 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:08:13.504 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:13.504 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:08:13.504 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:13.505 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:07:57.721034+0000", 2024-04-26T21:08:13.506 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.790+0000 7fd14b08b700 1 Processor -- start 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.790+0000 7fd14b08b700 1 -- start start 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.790+0000 7fd14b08b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd144107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.790+0000 7fd14b08b700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1441085b0 con 0x7fd144107bc0 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.790+0000 7fd148e27700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd144107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.790+0000 7fd148e27700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd144107fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51186/0 (socket says 172.21.15.129:51186) 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.790+0000 7fd148e27700 1 -- 172.21.15.129:0/1932042606 learned_addr learned my addr 172.21.15.129:0/1932042606 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.791+0000 7fd148e27700 1 -- 172.21.15.129:0/1932042606 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd144108dc0 con 0x7fd144107bc0 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.791+0000 7fd148e27700 1 --2- 172.21.15.129:0/1932042606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd144107fe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1380098a0 tx=0x7fd138009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6850fa9197d6e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.791+0000 7fd1437fe700 1 -- 172.21.15.129:0/1932042606 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd138004160 con 0x7fd144107bc0 2024-04-26T21:08:13.507 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.791+0000 7fd1437fe700 1 -- 172.21.15.129:0/1932042606 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd138004300 con 0x7fd144107bc0 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.791+0000 7fd1437fe700 1 -- 172.21.15.129:0/1932042606 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd138011af0 con 0x7fd144107bc0 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.792+0000 7fd14b08b700 1 -- 172.21.15.129:0/1932042606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 msgr2=0x7fd144107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.792+0000 7fd14b08b700 1 --2- 172.21.15.129:0/1932042606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd144107fe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1380098a0 tx=0x7fd138009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.792+0000 7fd14b08b700 1 -- 172.21.15.129:0/1932042606 shutdown_connections 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.792+0000 7fd14b08b700 1 --2- 172.21.15.129:0/1932042606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd144107fe0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.792+0000 7fd14b08b700 1 -- 172.21.15.129:0/1932042606 >> 172.21.15.129:0/1932042606 conn(0x7fd144103140 msgr2=0x7fd1441055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.792+0000 7fd14b08b700 1 -- 172.21.15.129:0/1932042606 shutdown_connections 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.792+0000 7fd14b08b700 1 -- 172.21.15.129:0/1932042606 wait complete. 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.792+0000 7fd14b08b700 1 Processor -- start 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.792+0000 7fd14b08b700 1 -- start start 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd14b08b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd14419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd14b08b700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410c170 con 0x7fd144107bc0 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd148e27700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd14419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:13.508 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd148e27700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd14419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51190/0 (socket says 172.21.15.129:51190) 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd148e27700 1 -- 172.21.15.129:0/1003050544 learned_addr learned my addr 172.21.15.129:0/1003050544 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd148e27700 1 -- 172.21.15.129:0/1003050544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14419d210 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd148e27700 1 --2- 172.21.15.129:0/1003050544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd14419ccd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1380117b0 tx=0x7fd138011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd141ffb700 1 -- 172.21.15.129:0/1003050544 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd138003af0 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd14b08b700 1 -- 172.21.15.129:0/1003050544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14419d4a0 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd141ffb700 1 -- 172.21.15.129:0/1003050544 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd138020070 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.793+0000 7fd141ffb700 1 -- 172.21.15.129:0/1003050544 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd138019630 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.794+0000 7fd14b08b700 1 -- 172.21.15.129:0/1003050544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1441a0180 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.794+0000 7fd141ffb700 1 -- 172.21.15.129:0/1003050544 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd138019c80 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.794+0000 7fd141ffb700 1 -- 172.21.15.129:0/1003050544 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd13800b720 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.794+0000 7fd14b08b700 1 -- 172.21.15.129:0/1003050544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd144064a80 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.798+0000 7fd141ffb700 1 -- 172.21.15.129:0/1003050544 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd1380042a0 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.869+0000 7fd14b08b700 1 -- 172.21.15.129:0/1003050544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd14402e6e0 con 0x7fd144107bc0 2024-04-26T21:08:13.509 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.869+0000 7fd141ffb700 1 -- 172.21.15.129:0/1003050544 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd13801a9c0 con 0x7fd144107bc0 2024-04-26T21:08:13.510 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.871+0000 7fd14b08b700 1 -- 172.21.15.129:0/1003050544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 msgr2=0x7fd14419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:13.510 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.871+0000 7fd14b08b700 1 --2- 172.21.15.129:0/1003050544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd14419ccd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1380117b0 tx=0x7fd138011b70 comp rx=0 tx=0).stop 2024-04-26T21:08:13.510 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.871+0000 7fd14b08b700 1 -- 172.21.15.129:0/1003050544 shutdown_connections 2024-04-26T21:08:13.510 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.871+0000 7fd14b08b700 1 --2- 172.21.15.129:0/1003050544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd144107bc0 0x7fd14419ccd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:13.510 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.871+0000 7fd14b08b700 1 -- 172.21.15.129:0/1003050544 >> 172.21.15.129:0/1003050544 conn(0x7fd144103140 msgr2=0x7fd144104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:13.510 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.871+0000 7fd14b08b700 1 -- 172.21.15.129:0/1003050544 shutdown_connections 2024-04-26T21:08:13.510 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:12.871+0000 7fd14b08b700 1 -- 172.21.15.129:0/1003050544 wait complete. 2024-04-26T21:08:13.510 INFO:teuthology.orchestra.run.smithi129.stdout:mgr not available, waiting (2/15)... 2024-04-26T21:08:13.916 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:13 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:13.635+0000 7f75ffd02200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:08:13.916 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:13 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:13.726+0000 7f75ffd02200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:08:14.198 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:13 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:13.972+0000 7f75ffd02200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:08:14.199 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:14 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:14.104+0000 7f75ffd02200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:08:14.199 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:14 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:14.197+0000 7f75ffd02200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:08:14.469 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:14 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:14.311+0000 7f75ffd02200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:14.970 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:14 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:14.614+0000 7f75ffd02200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:08:15.470 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:15.204+0000 7f75ffd02200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:08:15.470 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:15.416+0000 7f75ffd02200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:08:15.800 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: Activating manager daemon x 2024-04-26T21:08:15.800 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: mgrmap e2: x(active, starting, since 0.00315722s) 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: Manager daemon x is now available 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' 2024-04-26T21:08:15.801 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:15 smithi129 ceph-mon[29237]: from='mgr.14100 172.21.15.129:0/1507031239' entity='mgr.x' 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsid": "eccca2b2-0410-11ef-bc93-c7b262605968", 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:08:16.568 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:08:16.569 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:08:16.570 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:08:16.578 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:08:16.579 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:07:57.721034+0000", 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.895+0000 7fe9832c7700 1 Processor -- start 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe9832c7700 1 -- start start 2024-04-26T21:08:16.580 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe9832c7700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe9832c7700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c07ce70 con 0x7fe97c107560 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe981063700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe981063700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51256/0 (socket says 172.21.15.129:51256) 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe981063700 1 -- 172.21.15.129:0/2318905735 learned_addr learned my addr 172.21.15.129:0/2318905735 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe981063700 1 -- 172.21.15.129:0/2318905735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c107ec0 con 0x7fe97c107560 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe981063700 1 --2- 172.21.15.129:0/2318905735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c107980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9780098a0 tx=0x7fe978009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed41ddd54991e412 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe973fff700 1 -- 172.21.15.129:0/2318905735 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe978004160 con 0x7fe97c107560 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe973fff700 1 -- 172.21.15.129:0/2318905735 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe978004300 con 0x7fe97c107560 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.896+0000 7fe973fff700 1 -- 172.21.15.129:0/2318905735 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe978011af0 con 0x7fe97c107560 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.897+0000 7fe9832c7700 1 -- 172.21.15.129:0/2318905735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 msgr2=0x7fe97c107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.897+0000 7fe9832c7700 1 --2- 172.21.15.129:0/2318905735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c107980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9780098a0 tx=0x7fe978009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.897+0000 7fe9832c7700 1 -- 172.21.15.129:0/2318905735 shutdown_connections 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.897+0000 7fe9832c7700 1 --2- 172.21.15.129:0/2318905735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c107980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:16.581 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.897+0000 7fe9832c7700 1 -- 172.21.15.129:0/2318905735 >> 172.21.15.129:0/2318905735 conn(0x7fe97c103160 msgr2=0x7fe97c1055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:16.582 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.897+0000 7fe9832c7700 1 -- 172.21.15.129:0/2318905735 shutdown_connections 2024-04-26T21:08:16.582 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.897+0000 7fe9832c7700 1 -- 172.21.15.129:0/2318905735 wait complete. 2024-04-26T21:08:16.582 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.898+0000 7fe9832c7700 1 Processor -- start 2024-04-26T21:08:16.582 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.898+0000 7fe9832c7700 1 -- start start 2024-04-26T21:08:16.582 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.898+0000 7fe9832c7700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:16.582 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.898+0000 7fe9832c7700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c108ef0 con 0x7fe97c107560 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.898+0000 7fe981063700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.898+0000 7fe981063700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51270/0 (socket says 172.21.15.129:51270) 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.898+0000 7fe981063700 1 -- 172.21.15.129:0/3175123550 learned_addr learned my addr 172.21.15.129:0/3175123550 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.898+0000 7fe981063700 1 -- 172.21.15.129:0/3175123550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c19f2f0 con 0x7fe97c107560 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.898+0000 7fe981063700 1 --2- 172.21.15.129:0/3175123550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c19edb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe978011ac0 tx=0x7fe978011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.899+0000 7fe9727fc700 1 -- 172.21.15.129:0/3175123550 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe978003b20 con 0x7fe97c107560 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.899+0000 7fe9832c7700 1 -- 172.21.15.129:0/3175123550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c19f580 con 0x7fe97c107560 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.899+0000 7fe9727fc700 1 -- 172.21.15.129:0/3175123550 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe97801a8c0 con 0x7fe97c107560 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.899+0000 7fe9727fc700 1 -- 172.21.15.129:0/3175123550 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe978024870 con 0x7fe97c107560 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.899+0000 7fe9832c7700 1 -- 172.21.15.129:0/3175123550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c1a2270 con 0x7fe97c107560 2024-04-26T21:08:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.899+0000 7fe9727fc700 1 -- 172.21.15.129:0/3175123550 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7fe978022020 con 0x7fe97c107560 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.900+0000 7fe9727fc700 1 -- 172.21.15.129:0/3175123550 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe97804d110 con 0x7fe97c107560 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.900+0000 7fe9832c7700 1 -- 172.21.15.129:0/3175123550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97c064a80 con 0x7fe97c107560 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.902+0000 7fe9727fc700 1 -- 172.21.15.129:0/3175123550 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe97800fdd0 con 0x7fe97c107560 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.971+0000 7fe9832c7700 1 -- 172.21.15.129:0/3175123550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe97c02e590 con 0x7fe97c107560 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.972+0000 7fe9727fc700 1 -- 172.21.15.129:0/3175123550 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fe978024a10 con 0x7fe97c107560 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.972+0000 7fe9832c7700 1 -- 172.21.15.129:0/3175123550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 msgr2=0x7fe97c19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.972+0000 7fe9832c7700 1 --2- 172.21.15.129:0/3175123550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c19edb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe978011ac0 tx=0x7fe978011ba0 comp rx=0 tx=0).stop 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.972+0000 7fe9832c7700 1 -- 172.21.15.129:0/3175123550 shutdown_connections 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.972+0000 7fe9832c7700 1 --2- 172.21.15.129:0/3175123550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe97c107560 0x7fe97c19edb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.972+0000 7fe9832c7700 1 -- 172.21.15.129:0/3175123550 >> 172.21.15.129:0/3175123550 conn(0x7fe97c103160 msgr2=0x7fe97c195a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.972+0000 7fe9832c7700 1 -- 172.21.15.129:0/3175123550 shutdown_connections 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:15.972+0000 7fe9832c7700 1 -- 172.21.15.129:0/3175123550 wait complete. 2024-04-26T21:08:16.584 INFO:teuthology.orchestra.run.smithi129.stdout:mgr not available, waiting (3/15)... 2024-04-26T21:08:16.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:16 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3175123550' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:08:16.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:16 smithi129 ceph-mon[29237]: mgrmap e3: x(active, since 1.00645s) 2024-04-26T21:08:18.785 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:18 smithi129 ceph-mon[29237]: mgrmap e4: x(active, since 2s) 2024-04-26T21:08:19.708 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:19 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1637085806' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:08:19.815 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsid": "eccca2b2-0410-11ef-bc93-c7b262605968", 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:19.816 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:08:19.817 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:08:19.827 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:08:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:07:57.721034+0000", 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.985+0000 7f53d9ee4700 1 Processor -- start 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.985+0000 7f53d9ee4700 1 -- start start 2024-04-26T21:08:19.829 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.985+0000 7f53d9ee4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d41073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.985+0000 7f53d9ee4700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4107990 con 0x7f53d4106fa0 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.985+0000 7f53d37fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d41073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.985+0000 7f53d37fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d41073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51276/0 (socket says 172.21.15.129:51276) 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.985+0000 7f53d37fe700 1 -- 172.21.15.129:0/156457927 learned_addr learned my addr 172.21.15.129:0/156457927 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.985+0000 7f53d37fe700 1 -- 172.21.15.129:0/156457927 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d4107b10 con 0x7f53d4106fa0 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.985+0000 7f53d37fe700 1 --2- 172.21.15.129:0/156457927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d41073c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f53c80098a0 tx=0x7f53c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f3ced4a5528ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d27fc700 1 -- 172.21.15.129:0/156457927 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c8004160 con 0x7f53d4106fa0 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d27fc700 1 -- 172.21.15.129:0/156457927 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f53c8004300 con 0x7f53d4106fa0 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d27fc700 1 -- 172.21.15.129:0/156457927 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c8011af0 con 0x7f53d4106fa0 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d9ee4700 1 -- 172.21.15.129:0/156457927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 msgr2=0x7f53d41073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d9ee4700 1 --2- 172.21.15.129:0/156457927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d41073c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f53c80098a0 tx=0x7f53c8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d9ee4700 1 -- 172.21.15.129:0/156457927 shutdown_connections 2024-04-26T21:08:19.830 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d9ee4700 1 --2- 172.21.15.129:0/156457927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d41073c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d9ee4700 1 -- 172.21.15.129:0/156457927 >> 172.21.15.129:0/156457927 conn(0x7f53d4102960 msgr2=0x7f53d4104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d9ee4700 1 -- 172.21.15.129:0/156457927 shutdown_connections 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.986+0000 7f53d9ee4700 1 -- 172.21.15.129:0/156457927 wait complete. 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.987+0000 7f53d9ee4700 1 Processor -- start 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.987+0000 7f53d9ee4700 1 -- start start 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.987+0000 7f53d9ee4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.987+0000 7f53d9ee4700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f53d410ae00 con 0x7f53d4106fa0 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.987+0000 7f53d37fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.987+0000 7f53d37fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51278/0 (socket says 172.21.15.129:51278) 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.987+0000 7f53d37fe700 1 -- 172.21.15.129:0/1637085806 learned_addr learned my addr 172.21.15.129:0/1637085806 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.988+0000 7f53d37fe700 1 -- 172.21.15.129:0/1637085806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d419d0f0 con 0x7f53d4106fa0 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.988+0000 7f53d37fe700 1 --2- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d419cbb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53c8000c00 tx=0x7f53c8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.988+0000 7f53d0ff9700 1 -- 172.21.15.129:0/1637085806 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c8003b20 con 0x7f53d4106fa0 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.988+0000 7f53d0ff9700 1 -- 172.21.15.129:0/1637085806 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f53c801a8c0 con 0x7f53d4106fa0 2024-04-26T21:08:19.831 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.988+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d419d3e0 con 0x7f53d4106fa0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.988+0000 7f53d0ff9700 1 -- 172.21.15.129:0/1637085806 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c8024940 con 0x7f53d4106fa0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.988+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d41a0070 con 0x7f53d4106fa0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.989+0000 7f53d0ff9700 1 -- 172.21.15.129:0/1637085806 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f53c8022020 con 0x7f53d4106fa0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.989+0000 7f53d0ff9700 1 --2- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f53bc039560 0x7f53bc03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.989+0000 7f53d0ff9700 1 -- 172.21.15.129:0/1637085806 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53c801a430 con 0x7f53d4106fa0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.989+0000 7f53d2ffd700 1 --2- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f53bc039560 0x7f53bc03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.989+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d410d250 con 0x7f53d4106fa0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.990+0000 7f53d2ffd700 1 --2- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f53bc039560 0x7f53bc03ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53c4006fd0 tx=0x7f53c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:18.995+0000 7f53d0ff9700 1 -- 172.21.15.129:0/1637085806 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f53c802dc90 con 0x7f53d4106fa0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.277+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f53d41a05b0 con 0x7f53d4106fa0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.277+0000 7f53d0ff9700 1 -- 172.21.15.129:0/1637085806 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f53c800b3e0 con 0x7f53d4106fa0 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.279+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f53bc039560 msgr2=0x7f53bc03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.279+0000 7f53d9ee4700 1 --2- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f53bc039560 0x7f53bc03ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53c4006fd0 tx=0x7f53c4006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:19.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.279+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 msgr2=0x7f53d419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.279+0000 7f53d9ee4700 1 --2- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d419cbb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53c8000c00 tx=0x7f53c8011b70 comp rx=0 tx=0).stop 2024-04-26T21:08:19.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.280+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 shutdown_connections 2024-04-26T21:08:19.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.280+0000 7f53d9ee4700 1 --2- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f53bc039560 0x7f53bc03ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.280+0000 7f53d9ee4700 1 --2- 172.21.15.129:0/1637085806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53d4106fa0 0x7f53d419cbb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.280+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 >> 172.21.15.129:0/1637085806 conn(0x7f53d4102960 msgr2=0x7f53d41034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:19.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.280+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 shutdown_connections 2024-04-26T21:08:19.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:19.280+0000 7f53d9ee4700 1 -- 172.21.15.129:0/1637085806 wait complete. 2024-04-26T21:08:19.833 INFO:teuthology.orchestra.run.smithi129.stdout:mgr is available 2024-04-26T21:08:20.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:20 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3188301946' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:08:21.086 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:21.087 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:08:21.087 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout fsid = eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:08:21.087 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:08:21.087 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.274+0000 7fbbee4e2700 1 Processor -- start 2024-04-26T21:08:21.088 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.274+0000 7fbbee4e2700 1 -- start start 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.274+0000 7fbbee4e2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe807d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.274+0000 7fbbee4e2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe807d8f0 con 0x7fbbe807eec0 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.274+0000 7fbbe7fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe807d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.274+0000 7fbbe7fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe807d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51286/0 (socket says 172.21.15.129:51286) 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.274+0000 7fbbe7fff700 1 -- 172.21.15.129:0/3550890306 learned_addr learned my addr 172.21.15.129:0/3550890306 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.274+0000 7fbbe7fff700 1 -- 172.21.15.129:0/3550890306 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe807da70 con 0x7fbbe807eec0 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.274+0000 7fbbe7fff700 1 --2- 172.21.15.129:0/3550890306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe807d320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80098a0 tx=0x7fbbd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8edd461a892708 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbe6ffd700 1 -- 172.21.15.129:0/3550890306 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd8004160 con 0x7fbbe807eec0 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbe6ffd700 1 -- 172.21.15.129:0/3550890306 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbbd8004300 con 0x7fbbe807eec0 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbe6ffd700 1 -- 172.21.15.129:0/3550890306 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd8011af0 con 0x7fbbe807eec0 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3550890306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 msgr2=0x7fbbe807d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbee4e2700 1 --2- 172.21.15.129:0/3550890306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe807d320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80098a0 tx=0x7fbbd8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3550890306 shutdown_connections 2024-04-26T21:08:21.089 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbee4e2700 1 --2- 172.21.15.129:0/3550890306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe807d320 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3550890306 >> 172.21.15.129:0/3550890306 conn(0x7fbbe81042b0 msgr2=0x7fbbe81066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3550890306 shutdown_connections 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.275+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3550890306 wait complete. 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.276+0000 7fbbee4e2700 1 Processor -- start 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.276+0000 7fbbee4e2700 1 -- start start 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.276+0000 7fbbee4e2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.276+0000 7fbbee4e2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe810f260 con 0x7fbbe807eec0 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.276+0000 7fbbe7fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.276+0000 7fbbe7fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe819ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51294/0 (socket says 172.21.15.129:51294) 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.276+0000 7fbbe7fff700 1 -- 172.21.15.129:0/3188301946 learned_addr learned my addr 172.21.15.129:0/3188301946 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.277+0000 7fbbe7fff700 1 -- 172.21.15.129:0/3188301946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe819f380 con 0x7fbbe807eec0 2024-04-26T21:08:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.277+0000 7fbbe7fff700 1 --2- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe819ee40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fb0 tx=0x7fbbd8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.277+0000 7fbbe57fa700 1 -- 172.21.15.129:0/3188301946 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd80036a0 con 0x7fbbe807eec0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.277+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe819f610 con 0x7fbbe807eec0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.277+0000 7fbbe57fa700 1 -- 172.21.15.129:0/3188301946 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbbd801a430 con 0x7fbbe807eec0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.277+0000 7fbbe57fa700 1 -- 172.21.15.129:0/3188301946 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd8019550 con 0x7fbbe807eec0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.277+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81a2300 con 0x7fbbe807eec0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.278+0000 7fbbe57fa700 1 -- 172.21.15.129:0/3188301946 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fbbd8019850 con 0x7fbbe807eec0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.278+0000 7fbbe57fa700 1 --2- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fbbd0039510 0x7fbbd003b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.278+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe8064a80 con 0x7fbbe807eec0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.278+0000 7fbbe57fa700 1 -- 172.21.15.129:0/3188301946 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbbd804d8d0 con 0x7fbbe807eec0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.278+0000 7fbbe77fe700 1 --2- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fbbd0039510 0x7fbbd003b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.278+0000 7fbbe77fe700 1 --2- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fbbd0039510 0x7fbbd003b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc006fd0 tx=0x7fbbdc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.284+0000 7fbbe57fa700 1 -- 172.21.15.129:0/3188301946 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbbd804d2f0 con 0x7fbbe807eec0 2024-04-26T21:08:21.091 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.479+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbbe81a24d0 con 0x7fbbe807eec0 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.480+0000 7fbbe57fa700 1 -- 172.21.15.129:0/3188301946 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fbbd800fda0 con 0x7fbbe807eec0 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.482+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fbbd0039510 msgr2=0x7fbbd003b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.482+0000 7fbbee4e2700 1 --2- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fbbd0039510 0x7fbbd003b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc006fd0 tx=0x7fbbdc006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.482+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 msgr2=0x7fbbe819ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.482+0000 7fbbee4e2700 1 --2- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe819ee40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fb0 tx=0x7fbbd8011940 comp rx=0 tx=0).stop 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.482+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 shutdown_connections 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.482+0000 7fbbee4e2700 1 --2- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fbbd0039510 0x7fbbd003b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.482+0000 7fbbee4e2700 1 --2- 172.21.15.129:0/3188301946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe807eec0 0x7fbbe819ee40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.483+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 >> 172.21.15.129:0/3188301946 conn(0x7fbbe81042b0 msgr2=0x7fbbe8104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.483+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 shutdown_connections 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:20.483+0000 7fbbee4e2700 1 -- 172.21.15.129:0/3188301946 wait complete. 2024-04-26T21:08:21.092 INFO:teuthology.orchestra.run.smithi129.stdout:Enabling cephadm module... 2024-04-26T21:08:22.116 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:21 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/998083098' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T21:08:22.116 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:21 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: ignoring --setuser ceph since I am not root 2024-04-26T21:08:22.116 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:21 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: ignoring --setgroup ceph since I am not root 2024-04-26T21:08:22.116 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:21 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:21.962+0000 7f92c10c3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:08:22.391 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:22 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:22.147+0000 7f92c10c3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:22.391 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:22 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:22.249+0000 7f92c10c3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:08:22.399 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.482+0000 7fb085450700 1 Processor -- start 2024-04-26T21:08:22.399 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.482+0000 7fb085450700 1 -- start start 2024-04-26T21:08:22.399 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb085450700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb080107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb085450700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801085b0 con 0x7fb080107bc0 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb07effd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb080107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb07effd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb080107fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51306/0 (socket says 172.21.15.129:51306) 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb07effd700 1 -- 172.21.15.129:0/2847369683 learned_addr learned my addr 172.21.15.129:0/2847369683 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb07effd700 1 -- 172.21.15.129:0/2847369683 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb080108dc0 con 0x7fb080107bc0 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb07effd700 1 --2- 172.21.15.129:0/2847369683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb080107fe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb07000bb40 tx=0x7fb07000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ded7e808e7ccac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb07dffb700 1 -- 172.21.15.129:0/2847369683 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb07000dbe0 con 0x7fb080107bc0 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb07dffb700 1 -- 172.21.15.129:0/2847369683 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb0700044c0 con 0x7fb080107bc0 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.483+0000 7fb07dffb700 1 -- 172.21.15.129:0/2847369683 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb070010470 con 0x7fb080107bc0 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.484+0000 7fb085450700 1 -- 172.21.15.129:0/2847369683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 msgr2=0x7fb080107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.484+0000 7fb085450700 1 --2- 172.21.15.129:0/2847369683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb080107fe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb07000bb40 tx=0x7fb07000be50 comp rx=0 tx=0).stop 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.484+0000 7fb085450700 1 -- 172.21.15.129:0/2847369683 shutdown_connections 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.484+0000 7fb085450700 1 --2- 172.21.15.129:0/2847369683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb080107fe0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.484+0000 7fb085450700 1 -- 172.21.15.129:0/2847369683 >> 172.21.15.129:0/2847369683 conn(0x7fb080103140 msgr2=0x7fb0801055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:22.400 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.484+0000 7fb085450700 1 -- 172.21.15.129:0/2847369683 shutdown_connections 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.484+0000 7fb085450700 1 -- 172.21.15.129:0/2847369683 wait complete. 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.485+0000 7fb085450700 1 Processor -- start 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.485+0000 7fb085450700 1 -- start start 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.485+0000 7fb085450700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb08019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.485+0000 7fb085450700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb08007bfd0 con 0x7fb080107bc0 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.485+0000 7fb07effd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb08019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.485+0000 7fb07effd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb08019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51316/0 (socket says 172.21.15.129:51316) 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.485+0000 7fb07effd700 1 -- 172.21.15.129:0/998083098 learned_addr learned my addr 172.21.15.129:0/998083098 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.485+0000 7fb07effd700 1 -- 172.21.15.129:0/998083098 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08019f2f0 con 0x7fb080107bc0 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.485+0000 7fb07effd700 1 --2- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb08019edb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb07000b340 tx=0x7fb070003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.486+0000 7fb077fff700 1 -- 172.21.15.129:0/998083098 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb070010720 con 0x7fb080107bc0 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.486+0000 7fb077fff700 1 -- 172.21.15.129:0/998083098 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb07001d070 con 0x7fb080107bc0 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.486+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08019f580 con 0x7fb080107bc0 2024-04-26T21:08:22.401 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.486+0000 7fb077fff700 1 -- 172.21.15.129:0/998083098 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb070021ba0 con 0x7fb080107bc0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.486+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0801a2270 con 0x7fb080107bc0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.486+0000 7fb077fff700 1 -- 172.21.15.129:0/998083098 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fb07001f030 con 0x7fb080107bc0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.487+0000 7fb077fff700 1 --2- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fb06c039560 0x7fb06c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.487+0000 7fb077fff700 1 -- 172.21.15.129:0/998083098 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb07004d640 con 0x7fb080107bc0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.487+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb080064a80 con 0x7fb080107bc0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.487+0000 7fb07e7fc700 1 --2- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fb06c039560 0x7fb06c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.487+0000 7fb07e7fc700 1 --2- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fb06c039560 0x7fb06c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb068006fd0 tx=0x7fb068006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.492+0000 7fb077fff700 1 -- 172.21.15.129:0/998083098 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb070015070 con 0x7fb080107bc0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.746+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb080193f00 con 0x7fb080107bc0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.797+0000 7fb077fff700 1 -- 172.21.15.129:0/998083098 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb080193f00 con 0x7fb080107bc0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.799+0000 7fb077fff700 1 -- 172.21.15.129:0/998083098 <== mon.0 v2:172.21.15.129:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fb070019580 con 0x7fb080107bc0 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fb06c039560 msgr2=0x7fb06c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:22.402 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 --2- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fb06c039560 0x7fb06c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb068006fd0 tx=0x7fb068006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:22.403 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 msgr2=0x7fb08019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:22.403 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 --2- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb08019edb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb07000b340 tx=0x7fb070003dc0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.403 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 shutdown_connections 2024-04-26T21:08:22.403 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 --2- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fb06c039560 0x7fb06c03ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.403 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 --2- 172.21.15.129:0/998083098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb080107bc0 0x7fb08019edb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.403 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 >> 172.21.15.129:0/998083098 conn(0x7fb080103140 msgr2=0x7fb080190fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:22.403 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 shutdown_connections 2024-04-26T21:08:22.403 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:21.801+0000 7fb085450700 1 -- 172.21.15.129:0/998083098 wait complete. 2024-04-26T21:08:22.651 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:22 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:22.563+0000 7f92c10c3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:08:22.651 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:22 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:22.649+0000 7f92c10c3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:08:22.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:22 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/998083098' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T21:08:22.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:22 smithi129 ceph-mon[29237]: mgrmap e5: x(active, since 6s) 2024-04-26T21:08:23.740 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:08:23.740 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T21:08:23.740 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.789+0000 7f11e997d700 1 Processor -- start 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.789+0000 7f11e997d700 1 -- start start 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.790+0000 7f11e997d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e41095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.790+0000 7f11e997d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11e4109b90 con 0x7f11e41091a0 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.790+0000 7f11e2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e41095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.790+0000 7f11e2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e41095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49618/0 (socket says 172.21.15.129:49618) 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.790+0000 7f11e2ffd700 1 -- 172.21.15.129:0/1570383767 learned_addr learned my addr 172.21.15.129:0/1570383767 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.790+0000 7f11e2ffd700 1 -- 172.21.15.129:0/1570383767 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e4107db0 con 0x7f11e41091a0 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.790+0000 7f11e2ffd700 1 --2- 172.21.15.129:0/1570383767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e41095c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f11d40098a0 tx=0x7f11d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba22b7f59231ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:23.741 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.790+0000 7f11e1ffb700 1 -- 172.21.15.129:0/1570383767 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11d4004160 con 0x7f11e41091a0 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.791+0000 7f11e1ffb700 1 -- 172.21.15.129:0/1570383767 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f11d4004300 con 0x7f11e41091a0 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.791+0000 7f11e1ffb700 1 -- 172.21.15.129:0/1570383767 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11d4011af0 con 0x7f11e41091a0 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.791+0000 7f11e997d700 1 -- 172.21.15.129:0/1570383767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 msgr2=0x7f11e41095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.791+0000 7f11e997d700 1 --2- 172.21.15.129:0/1570383767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e41095c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f11d40098a0 tx=0x7f11d4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.791+0000 7f11e997d700 1 -- 172.21.15.129:0/1570383767 shutdown_connections 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.791+0000 7f11e997d700 1 --2- 172.21.15.129:0/1570383767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e41095c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.791+0000 7f11e997d700 1 -- 172.21.15.129:0/1570383767 >> 172.21.15.129:0/1570383767 conn(0x7f11e41029a0 msgr2=0x7f11e4104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.791+0000 7f11e997d700 1 -- 172.21.15.129:0/1570383767 shutdown_connections 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.791+0000 7f11e997d700 1 -- 172.21.15.129:0/1570383767 wait complete. 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.792+0000 7f11e997d700 1 Processor -- start 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.792+0000 7f11e997d700 1 -- start start 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.792+0000 7f11e997d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:23.742 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.792+0000 7f11e997d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11e410c170 con 0x7f11e41091a0 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.792+0000 7f11e2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.792+0000 7f11e2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49632/0 (socket says 172.21.15.129:49632) 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.792+0000 7f11e2ffd700 1 -- 172.21.15.129:0/2076764920 learned_addr learned my addr 172.21.15.129:0/2076764920 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.793+0000 7f11e2ffd700 1 -- 172.21.15.129:0/2076764920 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e419d190 con 0x7f11e41091a0 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.793+0000 7f11e2ffd700 1 --2- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e419cc50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f11d4003f40 tx=0x7f11d4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.793+0000 7f11e897b700 1 -- 172.21.15.129:0/2076764920 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11d40038b0 con 0x7f11e41091a0 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.793+0000 7f11e897b700 1 -- 172.21.15.129:0/2076764920 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f11d4003a50 con 0x7f11e41091a0 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.793+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e419d420 con 0x7f11e41091a0 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.793+0000 7f11e897b700 1 -- 172.21.15.129:0/2076764920 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11d401a5c0 con 0x7f11e41091a0 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.793+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e41a0110 con 0x7f11e41091a0 2024-04-26T21:08:23.743 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.794+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e4064a80 con 0x7f11e41091a0 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.794+0000 7f11e897b700 1 -- 172.21.15.129:0/2076764920 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f11d401a8c0 con 0x7f11e41091a0 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.794+0000 7f11e897b700 1 --2- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f11cc039560 0x7f11cc03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.794+0000 7f11e897b700 1 -- 172.21.15.129:0/2076764920 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f11d404db60 con 0x7f11e41091a0 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.794+0000 7f11e27fc700 1 -- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f11cc039560 msgr2=0x7f11cc03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/1433037991 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.794+0000 7f11e27fc700 1 --2- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f11cc039560 0x7f11cc03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.799+0000 7f11e897b700 1 -- 172.21.15.129:0/2076764920 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f11d4019c70 con 0x7f11e41091a0 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.995+0000 7f11e27fc700 1 -- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f11cc039560 msgr2=0x7f11cc03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/1433037991 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:22.995+0000 7f11e27fc700 1 --2- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f11cc039560 0x7f11cc03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.090+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f11e40008d0 con 0x7f11e41091a0 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.092+0000 7f11e897b700 1 -- 172.21.15.129:0/2076764920 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f11d404d580 con 0x7f11e41091a0 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f11cc039560 msgr2=0x7f11cc03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 --2- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f11cc039560 0x7f11cc03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 msgr2=0x7f11e419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:23.744 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 --2- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e419cc50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f11d4003f40 tx=0x7f11d4011890 comp rx=0 tx=0).stop 2024-04-26T21:08:23.745 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 shutdown_connections 2024-04-26T21:08:23.745 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 --2- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7f11cc039560 0x7f11cc03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:23.745 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 --2- 172.21.15.129:0/2076764920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11e41091a0 0x7f11e419cc50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:23.745 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 >> 172.21.15.129:0/2076764920 conn(0x7f11e41029a0 msgr2=0x7f11e4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:23.745 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 shutdown_connections 2024-04-26T21:08:23.745 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:23.095+0000 7f11e997d700 1 -- 172.21.15.129:0/2076764920 wait complete. 2024-04-26T21:08:23.745 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for the mgr to restart... 2024-04-26T21:08:23.745 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mgr epoch 5... 2024-04-26T21:08:23.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:23 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2076764920' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:08:23.984 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:23 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:23.884+0000 7f92c10c3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:08:24.290 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:24.058+0000 7f92c10c3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:24.290 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:24.231+0000 7f92c10c3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:08:24.611 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:24.518+0000 7f92c10c3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:08:24.612 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:24.610+0000 7f92c10c3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:08:24.969 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:24.815+0000 7f92c10c3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:08:25.469 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:25 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:25.006+0000 7f92c10c3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:25.970 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:25 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:25.604+0000 7f92c10c3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:08:26.316 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:26.019+0000 7f92c10c3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:08:26.316 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:26.115+0000 7f92c10c3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:08:26.316 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:26.314+0000 7f92c10c3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:08:27.193 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:26.862+0000 7f92c10c3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:08:27.193 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:26.952+0000 7f92c10c3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:08:27.470 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:27.191+0000 7f92c10c3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:08:27.470 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:27.323+0000 7f92c10c3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:08:27.470 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:27.417+0000 7f92c10c3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:08:27.846 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:27.530+0000 7f92c10c3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:27.847 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:27.845+0000 7f92c10c3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:08:28.707 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:28.439+0000 7f92c10c3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:08:28.707 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:28.653+0000 7f92c10c3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: Active manager daemon x restarted 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: Activating manager daemon x 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: mgrmap e6: x(active, starting, since 0.0037016s) 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: Manager daemon x is now available 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:08:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:28 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:08:29.954 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:29 smithi129 ceph-mon[29237]: Found migration_current of "None". Setting to last migration. 2024-04-26T21:08:29.955 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:29 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:29.955 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:29 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:29.955 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:29 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:29.955 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:29 smithi129 ceph-mon[29237]: mgrmap e7: x(active, since 1.00606s) 2024-04-26T21:08:30.183 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:08:30.183 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T21:08:30.183 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:08:30.183 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:08:30.183 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.083+0000 7fc1df643700 1 Processor -- start 2024-04-26T21:08:30.183 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.083+0000 7fc1df643700 1 -- start start 2024-04-26T21:08:30.183 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1df643700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d807d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1df643700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d807d8f0 con 0x7fc1d807eec0 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1dd3df700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d807d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1dd3df700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d807d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49646/0 (socket says 172.21.15.129:49646) 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1dd3df700 1 -- 172.21.15.129:0/1029444490 learned_addr learned my addr 172.21.15.129:0/1029444490 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1dd3df700 1 -- 172.21.15.129:0/1029444490 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d807da70 con 0x7fc1d807eec0 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1dd3df700 1 --2- 172.21.15.129:0/1029444490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d807d320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80098a0 tx=0x7fc1c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb5858e4dd45f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1cffff700 1 -- 172.21.15.129:0/1029444490 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1c8004160 con 0x7fc1d807eec0 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1cffff700 1 -- 172.21.15.129:0/1029444490 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc1c8004300 con 0x7fc1d807eec0 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.084+0000 7fc1cffff700 1 -- 172.21.15.129:0/1029444490 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1c8011af0 con 0x7fc1d807eec0 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.085+0000 7fc1df643700 1 -- 172.21.15.129:0/1029444490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 msgr2=0x7fc1d807d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.085+0000 7fc1df643700 1 --2- 172.21.15.129:0/1029444490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d807d320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80098a0 tx=0x7fc1c8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.085+0000 7fc1df643700 1 -- 172.21.15.129:0/1029444490 shutdown_connections 2024-04-26T21:08:30.184 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.085+0000 7fc1df643700 1 --2- 172.21.15.129:0/1029444490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d807d320 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.085+0000 7fc1df643700 1 -- 172.21.15.129:0/1029444490 >> 172.21.15.129:0/1029444490 conn(0x7fc1d8104280 msgr2=0x7fc1d81066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.085+0000 7fc1df643700 1 -- 172.21.15.129:0/1029444490 shutdown_connections 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.085+0000 7fc1df643700 1 -- 172.21.15.129:0/1029444490 wait complete. 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.086+0000 7fc1df643700 1 Processor -- start 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.086+0000 7fc1df643700 1 -- start start 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.086+0000 7fc1df643700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.086+0000 7fc1df643700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d810f250 con 0x7fc1d807eec0 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.086+0000 7fc1dd3df700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.086+0000 7fc1dd3df700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49654/0 (socket says 172.21.15.129:49654) 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.086+0000 7fc1dd3df700 1 -- 172.21.15.129:0/604593313 learned_addr learned my addr 172.21.15.129:0/604593313 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.086+0000 7fc1dd3df700 1 -- 172.21.15.129:0/604593313 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d819f340 con 0x7fc1d807eec0 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.086+0000 7fc1dd3df700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d819ee00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8003f40 tx=0x7fc1c8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:30.185 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.087+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1c80038b0 con 0x7fc1d807eec0 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.087+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc1c8003a50 con 0x7fc1d807eec0 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.087+0000 7fc1df643700 1 -- 172.21.15.129:0/604593313 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d819f5d0 con 0x7fc1d807eec0 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.087+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1c801a5c0 con 0x7fc1d807eec0 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.087+0000 7fc1df643700 1 -- 172.21.15.129:0/604593313 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d81a22c0 con 0x7fc1d807eec0 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.088+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fc1c801a870 con 0x7fc1d807eec0 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.088+0000 7fc1ce7fc700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 0x7fc1c403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.088+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 --> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc1c403c180 con 0x7fc1c40395b0 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.088+0000 7fc1dcbde700 1 -- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 msgr2=0x7fc1c403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/1433037991 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.088+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc1c804da70 con 0x7fc1d807eec0 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.088+0000 7fc1dcbde700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 0x7fc1c403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.289+0000 7fc1dcbde700 1 -- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 msgr2=0x7fc1c403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/1433037991 2024-04-26T21:08:30.186 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.289+0000 7fc1dcbde700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 0x7fc1c403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.690+0000 7fc1dcbde700 1 -- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 msgr2=0x7fc1c403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/1433037991 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:24.690+0000 7fc1dcbde700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 0x7fc1c403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:25.491+0000 7fc1dcbde700 1 -- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 msgr2=0x7fc1c403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/1433037991 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:25.491+0000 7fc1dcbde700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 0x7fc1c403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:27.092+0000 7fc1dcbde700 1 -- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 msgr2=0x7fc1c403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/1433037991 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:27.092+0000 7fc1dcbde700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 0x7fc1c403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:28.657+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fc1c801abb0 con 0x7fc1d807eec0 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:28.657+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 msgr2=0x7fc1c403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:28.657+0000 7fc1ce7fc700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 0x7fc1c403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.658+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fc1c804f910 con 0x7fc1d807eec0 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.658+0000 7fc1ce7fc700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fc1c403cea0 0x7fc1c403f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.658+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc1c403c180 con 0x7fc1c403cea0 2024-04-26T21:08:30.187 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.659+0000 7fc1dcbde700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fc1c403cea0 0x7fc1c403f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.659+0000 7fc1dcbde700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fc1c403cea0 0x7fc1c403f290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4003a10 tx=0x7fc1d40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.660+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 <== mgr.14120 v2:172.21.15.129:6800/948723446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc1c403c180 con 0x7fc1c403cea0 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.668+0000 7fc1df643700 1 -- 172.21.15.129:0/604593313 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc1d806a520 con 0x7fc1c403cea0 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.668+0000 7fc1ce7fc700 1 -- 172.21.15.129:0/604593313 <== mgr.14120 v2:172.21.15.129:6800/948723446 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fc1d806a520 con 0x7fc1c403cea0 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.668+0000 7fc1df643700 1 -- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fc1c403cea0 msgr2=0x7fc1c403f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.668+0000 7fc1df643700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fc1c403cea0 0x7fc1c403f290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4003a10 tx=0x7fc1d40092b0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.668+0000 7fc1df643700 1 -- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 msgr2=0x7fc1d819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.668+0000 7fc1df643700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d819ee00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8003f40 tx=0x7fc1c8011890 comp rx=0 tx=0).stop 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.669+0000 7fc1df643700 1 -- 172.21.15.129:0/604593313 shutdown_connections 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.669+0000 7fc1df643700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fc1c403cea0 0x7fc1c403f290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.669+0000 7fc1df643700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1d807eec0 0x7fc1d819ee00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.669+0000 7fc1df643700 1 --2- 172.21.15.129:0/604593313 >> [v2:172.21.15.129:6800/1433037991,v1:172.21.15.129:6801/1433037991] conn(0x7fc1c40395b0 0x7fc1c403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.669+0000 7fc1df643700 1 -- 172.21.15.129:0/604593313 >> 172.21.15.129:0/604593313 conn(0x7fc1d8104280 msgr2=0x7fc1d8104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.669+0000 7fc1df643700 1 -- 172.21.15.129:0/604593313 shutdown_connections 2024-04-26T21:08:30.188 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:29.669+0000 7fc1df643700 1 -- 172.21.15.129:0/604593313 wait complete. 2024-04-26T21:08:30.189 INFO:teuthology.orchestra.run.smithi129.stdout:mgr epoch 5 is available 2024-04-26T21:08:30.189 INFO:teuthology.orchestra.run.smithi129.stdout:Setting orchestrator backend to cephadm... 2024-04-26T21:08:30.899 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:30 smithi129 ceph-mon[29237]: [26/Apr/2024:21:08:29] ENGINE Bus STARTING 2024-04-26T21:08:30.899 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:30 smithi129 ceph-mon[29237]: [26/Apr/2024:21:08:29] ENGINE Serving on https://172.21.15.129:7150 2024-04-26T21:08:30.899 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:30 smithi129 ceph-mon[29237]: [26/Apr/2024:21:08:29] ENGINE Serving on http://172.21.15.129:8765 2024-04-26T21:08:30.899 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:30 smithi129 ceph-mon[29237]: [26/Apr/2024:21:08:29] ENGINE Bus STARTED 2024-04-26T21:08:30.900 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:30 smithi129 ceph-mon[29237]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:08:30.900 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:30 smithi129 ceph-mon[29237]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:08:31.497 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.668+0000 7fe8fcfde700 1 Processor -- start 2024-04-26T21:08:31.497 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.668+0000 7fe8fcfde700 1 -- start start 2024-04-26T21:08:31.497 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.668+0000 7fe8fcfde700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f8107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:31.497 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.668+0000 7fe8fcfde700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f807ce30 con 0x7fe8f8107560 2024-04-26T21:08:31.497 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.669+0000 7fe8f659c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f8107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:31.497 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.669+0000 7fe8f659c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f8107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49740/0 (socket says 172.21.15.129:49740) 2024-04-26T21:08:31.497 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.669+0000 7fe8f659c700 1 -- 172.21.15.129:0/1067283725 learned_addr learned my addr 172.21.15.129:0/1067283725 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.669+0000 7fe8f659c700 1 -- 172.21.15.129:0/1067283725 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f8107ec0 con 0x7fe8f8107560 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.669+0000 7fe8f659c700 1 --2- 172.21.15.129:0/1067283725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f8107980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80098a0 tx=0x7fe8e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1858750a4f3754 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.669+0000 7fe8f5d9b700 1 -- 172.21.15.129:0/1067283725 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8e8004160 con 0x7fe8f8107560 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.669+0000 7fe8f5d9b700 1 -- 172.21.15.129:0/1067283725 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe8e8004300 con 0x7fe8f8107560 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.669+0000 7fe8f5d9b700 1 -- 172.21.15.129:0/1067283725 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8e8011af0 con 0x7fe8f8107560 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.670+0000 7fe8fcfde700 1 -- 172.21.15.129:0/1067283725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 msgr2=0x7fe8f8107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.670+0000 7fe8fcfde700 1 --2- 172.21.15.129:0/1067283725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f8107980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80098a0 tx=0x7fe8e8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.670+0000 7fe8fcfde700 1 -- 172.21.15.129:0/1067283725 shutdown_connections 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.670+0000 7fe8fcfde700 1 --2- 172.21.15.129:0/1067283725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f8107980 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.670+0000 7fe8fcfde700 1 -- 172.21.15.129:0/1067283725 >> 172.21.15.129:0/1067283725 conn(0x7fe8f8103140 msgr2=0x7fe8f8105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.670+0000 7fe8fcfde700 1 -- 172.21.15.129:0/1067283725 shutdown_connections 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.670+0000 7fe8fcfde700 1 -- 172.21.15.129:0/1067283725 wait complete. 2024-04-26T21:08:31.498 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.670+0000 7fe8fcfde700 1 Processor -- start 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8fcfde700 1 -- start start 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8fcfde700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8fcfde700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8108ef0 con 0x7fe8f8107560 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8f659c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8f659c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f819ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49750/0 (socket says 172.21.15.129:49750) 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8f659c700 1 -- 172.21.15.129:0/695066869 learned_addr learned my addr 172.21.15.129:0/695066869 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8f659c700 1 -- 172.21.15.129:0/695066869 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f819f370 con 0x7fe8f8107560 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8f659c700 1 --2- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f819ee30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8003f40 tx=0x7fe8e8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8effff700 1 -- 172.21.15.129:0/695066869 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8e80038b0 con 0x7fe8f8107560 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f819f600 con 0x7fe8f8107560 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f81a22f0 con 0x7fe8f8107560 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8effff700 1 -- 172.21.15.129:0/695066869 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe8e8004300 con 0x7fe8f8107560 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.671+0000 7fe8effff700 1 -- 172.21.15.129:0/695066869 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8e8024c10 con 0x7fe8f8107560 2024-04-26T21:08:31.499 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.672+0000 7fe8effff700 1 -- 172.21.15.129:0/695066869 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fe8e801a430 con 0x7fe8f8107560 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.672+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d8005360 con 0x7fe8f8107560 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.672+0000 7fe8effff700 1 --2- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fe8e4039450 0x7fe8e403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.673+0000 7fe8effff700 1 -- 172.21.15.129:0/695066869 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe8e804d860 con 0x7fe8f8107560 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.673+0000 7fe8edd9b700 1 --2- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fe8e4039450 0x7fe8e403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.673+0000 7fe8edd9b700 1 --2- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fe8e4039450 0x7fe8e403b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0006fd0 tx=0x7fe8e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.678+0000 7fe8effff700 1 -- 172.21.15.129:0/695066869 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe8e8023390 con 0x7fe8f8107560 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.714+0000 7fe8effff700 1 -- 172.21.15.129:0/695066869 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe8e804fd10 con 0x7fe8f8107560 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.898+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fe8d8000bf0 con 0x7fe8e4039450 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.903+0000 7fe8effff700 1 -- 172.21.15.129:0/695066869 <== mgr.14120 v2:172.21.15.129:6800/948723446 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8d8000bf0 con 0x7fe8e4039450 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fe8e4039450 msgr2=0x7fe8e403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 --2- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fe8e4039450 0x7fe8e403b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0006fd0 tx=0x7fe8e0006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 msgr2=0x7fe8f819ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 --2- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f819ee30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8003f40 tx=0x7fe8e8011890 comp rx=0 tx=0).stop 2024-04-26T21:08:31.500 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 shutdown_connections 2024-04-26T21:08:31.501 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 --2- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fe8e4039450 0x7fe8e403b910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:31.501 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 --2- 172.21.15.129:0/695066869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8f8107560 0x7fe8f819ee30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:31.501 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 >> 172.21.15.129:0/695066869 conn(0x7fe8f8103140 msgr2=0x7fe8f8195a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:31.501 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 shutdown_connections 2024-04-26T21:08:31.501 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:30.906+0000 7fe8fcfde700 1 -- 172.21.15.129:0/695066869 wait complete. 2024-04-26T21:08:31.823 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:31 smithi129 ceph-mon[29237]: mgrmap e8: x(active, since 2s) 2024-04-26T21:08:31.823 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:31 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:31.823 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:31 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:32.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:32 smithi129 ceph-mon[29237]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:32.810 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T21:08:32.810 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.967+0000 7f73e4014700 1 Processor -- start 2024-04-26T21:08:32.810 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.967+0000 7f73e4014700 1 -- start start 2024-04-26T21:08:32.810 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.967+0000 7f73e4014700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc1090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:32.810 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.967+0000 7f73e4014700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1096c0 con 0x7f73dc108cd0 2024-04-26T21:08:32.810 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.967+0000 7f73e1db0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc1090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:32.810 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.968+0000 7f73e1db0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc1090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49766/0 (socket says 172.21.15.129:49766) 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.968+0000 7f73e1db0700 1 -- 172.21.15.129:0/3763858765 learned_addr learned my addr 172.21.15.129:0/3763858765 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.968+0000 7f73e1db0700 1 -- 172.21.15.129:0/3763858765 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc109ed0 con 0x7f73dc108cd0 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.968+0000 7f73e1db0700 1 --2- 172.21.15.129:0/3763858765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc1090f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f73d00098a0 tx=0x7f73d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b07bbb2c54a3eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.968+0000 7f73e0dae700 1 -- 172.21.15.129:0/3763858765 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73d0004160 con 0x7f73dc108cd0 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.968+0000 7f73e0dae700 1 -- 172.21.15.129:0/3763858765 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f73d0004300 con 0x7f73dc108cd0 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.968+0000 7f73e0dae700 1 -- 172.21.15.129:0/3763858765 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73d0011af0 con 0x7f73dc108cd0 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.969+0000 7f73e4014700 1 -- 172.21.15.129:0/3763858765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 msgr2=0x7f73dc1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.969+0000 7f73e4014700 1 --2- 172.21.15.129:0/3763858765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc1090f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f73d00098a0 tx=0x7f73d0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.969+0000 7f73e4014700 1 -- 172.21.15.129:0/3763858765 shutdown_connections 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.969+0000 7f73e4014700 1 --2- 172.21.15.129:0/3763858765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc1090f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.969+0000 7f73e4014700 1 -- 172.21.15.129:0/3763858765 >> 172.21.15.129:0/3763858765 conn(0x7f73dc1042b0 msgr2=0x7f73dc1066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.969+0000 7f73e4014700 1 -- 172.21.15.129:0/3763858765 shutdown_connections 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.969+0000 7f73e4014700 1 -- 172.21.15.129:0/3763858765 wait complete. 2024-04-26T21:08:32.811 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.969+0000 7f73e4014700 1 Processor -- start 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e4014700 1 -- start start 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e4014700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e4014700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc10d280 con 0x7f73dc108cd0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e1db0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e1db0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49768/0 (socket says 172.21.15.129:49768) 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e1db0700 1 -- 172.21.15.129:0/3200491563 learned_addr learned my addr 172.21.15.129:0/3200491563 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e1db0700 1 -- 172.21.15.129:0/3200491563 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc19d1c0 con 0x7f73dc108cd0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e1db0700 1 --2- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc19cc80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73d00117b0 tx=0x7f73d0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73ceffd700 1 -- 172.21.15.129:0/3200491563 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73d0003af0 con 0x7f73dc108cd0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73ceffd700 1 -- 172.21.15.129:0/3200491563 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f73d0020070 con 0x7f73dc108cd0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc19d450 con 0x7f73dc108cd0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73ceffd700 1 -- 172.21.15.129:0/3200491563 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73d0019630 con 0x7f73dc108cd0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.970+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc1a0140 con 0x7f73dc108cd0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.971+0000 7f73ceffd700 1 -- 172.21.15.129:0/3200491563 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f73d001a9c0 con 0x7f73dc108cd0 2024-04-26T21:08:32.812 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.971+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73dc06a520 con 0x7f73dc108cd0 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.971+0000 7f73ceffd700 1 --2- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f73c80395b0 0x7f73c803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.972+0000 7f73ceffd700 1 -- 172.21.15.129:0/3200491563 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f73d004db40 con 0x7f73dc108cd0 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.972+0000 7f73e15af700 1 --2- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f73c80395b0 0x7f73c803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.972+0000 7f73e15af700 1 --2- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f73c80395b0 0x7f73c803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006fd0 tx=0x7f73d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:31.977+0000 7f73ceffd700 1 -- 172.21.15.129:0/3200491563 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f73d000e260 con 0x7f73dc108cd0 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.176+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f73dc105520 con 0x7f73c80395b0 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.177+0000 7f73ceffd700 1 -- 172.21.15.129:0/3200491563 <== mgr.14120 v2:172.21.15.129:6800/948723446 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f73dc105520 con 0x7f73c80395b0 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f73c80395b0 msgr2=0x7f73c803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 --2- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f73c80395b0 0x7f73c803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006fd0 tx=0x7f73d8006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 msgr2=0x7f73dc19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 --2- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc19cc80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73d00117b0 tx=0x7f73d0011b70 comp rx=0 tx=0).stop 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 shutdown_connections 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 --2- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f73c80395b0 0x7f73c803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 --2- 172.21.15.129:0/3200491563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73dc108cd0 0x7f73dc19cc80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:32.814 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 >> 172.21.15.129:0/3200491563 conn(0x7f73dc1042b0 msgr2=0x7f73dc104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:32.814 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 shutdown_connections 2024-04-26T21:08:32.814 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:32.179+0000 7f73e4014700 1 -- 172.21.15.129:0/3200491563 wait complete. 2024-04-26T21:08:32.814 INFO:teuthology.orchestra.run.smithi129.stdout:Generating ssh key... 2024-04-26T21:08:33.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-mon[29237]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:33.970 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: Generating public/private rsa key pair. 2024-04-26T21:08:33.970 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: Your identification has been saved in /tmp/tmpd7fx3s3s/key. 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: Your public key has been saved in /tmp/tmpd7fx3s3s/key.pub. 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: The key fingerprint is: 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: SHA256:zIklymIigZwCRqmopSy+bYr3SzqK9YBehTaNe997htc ceph-eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: The key's randomart image is: 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: +---[RSA 3072]----+ 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: |oo. | 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: |=.. | 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: |*o . . | 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: |+...+. * . | 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: |=oo=oo. S | 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: |==..+ | 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: |+ oo.. . . | 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: |+o=*. . .. + E | 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: |+*=++. . o= | 2024-04-26T21:08:33.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: +----[SHA256]-----+ 2024-04-26T21:08:34.375 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.237+0000 7f0723c31700 1 Processor -- start 2024-04-26T21:08:34.375 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.238+0000 7f0723c31700 1 -- start start 2024-04-26T21:08:34.375 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.238+0000 7f0723c31700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.375 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.238+0000 7f0723c31700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1085b0 con 0x7f071c107bc0 2024-04-26T21:08:34.375 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.238+0000 7f07219cd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.375 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.238+0000 7f07219cd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c107fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43132/0 (socket says 172.21.15.129:43132) 2024-04-26T21:08:34.375 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.238+0000 7f07219cd700 1 -- 172.21.15.129:0/3892019808 learned_addr learned my addr 172.21.15.129:0/3892019808 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:34.375 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.238+0000 7f07219cd700 1 -- 172.21.15.129:0/3892019808 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c108dc0 con 0x7f071c107bc0 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.239+0000 7f07219cd700 1 --2- 172.21.15.129:0/3892019808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c107fe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f071800bb40 tx=0x7f071800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea24f0209591cd34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.239+0000 7f07209cb700 1 -- 172.21.15.129:0/3892019808 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f071800dbe0 con 0x7f071c107bc0 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.239+0000 7f07209cb700 1 -- 172.21.15.129:0/3892019808 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f07180044c0 con 0x7f071c107bc0 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.239+0000 7f07209cb700 1 -- 172.21.15.129:0/3892019808 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0718010470 con 0x7f071c107bc0 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.239+0000 7f0723c31700 1 -- 172.21.15.129:0/3892019808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 msgr2=0x7f071c107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.239+0000 7f0723c31700 1 --2- 172.21.15.129:0/3892019808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c107fe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f071800bb40 tx=0x7f071800be50 comp rx=0 tx=0).stop 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.239+0000 7f0723c31700 1 -- 172.21.15.129:0/3892019808 shutdown_connections 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.239+0000 7f0723c31700 1 --2- 172.21.15.129:0/3892019808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c107fe0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.239+0000 7f0723c31700 1 -- 172.21.15.129:0/3892019808 >> 172.21.15.129:0/3892019808 conn(0x7f071c103140 msgr2=0x7f071c1055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.240+0000 7f0723c31700 1 -- 172.21.15.129:0/3892019808 shutdown_connections 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.240+0000 7f0723c31700 1 -- 172.21.15.129:0/3892019808 wait complete. 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.240+0000 7f0723c31700 1 Processor -- start 2024-04-26T21:08:34.376 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.240+0000 7f0723c31700 1 -- start start 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.240+0000 7f0723c31700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.240+0000 7f0723c31700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f071c07bfd0 con 0x7f071c107bc0 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.240+0000 7f07219cd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.240+0000 7f07219cd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43146/0 (socket says 172.21.15.129:43146) 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.240+0000 7f07219cd700 1 -- 172.21.15.129:0/1603043702 learned_addr learned my addr 172.21.15.129:0/1603043702 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.241+0000 7f07219cd700 1 -- 172.21.15.129:0/1603043702 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c19f310 con 0x7f071c107bc0 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.241+0000 7f07219cd700 1 --2- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c19edd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f071800b340 tx=0x7f0718003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.241+0000 7f0712ffd700 1 -- 172.21.15.129:0/1603043702 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0718010720 con 0x7f071c107bc0 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.241+0000 7f0712ffd700 1 -- 172.21.15.129:0/1603043702 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f071801d070 con 0x7f071c107bc0 2024-04-26T21:08:34.377 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.241+0000 7f0712ffd700 1 -- 172.21.15.129:0/1603043702 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0718021ba0 con 0x7f071c107bc0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.241+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071c19f5a0 con 0x7f071c107bc0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.241+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071c1a2290 con 0x7f071c107bc0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.242+0000 7f0712ffd700 1 -- 172.21.15.129:0/1603043702 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f071801f030 con 0x7f071c107bc0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.242+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071c064a80 con 0x7f071c107bc0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.242+0000 7f0712ffd700 1 --2- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f07080395b0 0x7f070803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.242+0000 7f0712ffd700 1 -- 172.21.15.129:0/1603043702 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f071804d970 con 0x7f071c107bc0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.242+0000 7f07211cc700 1 --2- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f07080395b0 0x7f070803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.243+0000 7f07211cc700 1 --2- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f07080395b0 0x7f070803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f070c006fd0 tx=0x7f070c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.251+0000 7f0712ffd700 1 -- 172.21.15.129:0/1603043702 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0718015070 con 0x7f071c107bc0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.457+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f071c063860 con 0x7f07080395b0 2024-04-26T21:08:34.378 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.797+0000 7f0712ffd700 1 -- 172.21.15.129:0/1603043702 <== mgr.14120 v2:172.21.15.129:6800/948723446 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f071c063860 con 0x7f07080395b0 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.799+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f07080395b0 msgr2=0x7f070803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.799+0000 7f0723c31700 1 --2- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f07080395b0 0x7f070803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f070c006fd0 tx=0x7f070c006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.799+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 msgr2=0x7f071c19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.799+0000 7f0723c31700 1 --2- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c19edd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f071800b340 tx=0x7f0718003dc0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.799+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 shutdown_connections 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.799+0000 7f0723c31700 1 --2- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f07080395b0 0x7f070803ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.799+0000 7f0723c31700 1 --2- 172.21.15.129:0/1603043702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f071c107bc0 0x7f071c19edd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.799+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 >> 172.21.15.129:0/1603043702 conn(0x7f071c103140 msgr2=0x7f071c1953f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.800+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 shutdown_connections 2024-04-26T21:08:34.379 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:33.800+0000 7f0723c31700 1 -- 172.21.15.129:0/1603043702 wait complete. 2024-04-26T21:08:34.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:34 smithi129 ceph-mon[29237]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:34.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:34 smithi129 ceph-mon[29237]: Generating ssh key... 2024-04-26T21:08:34.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:34 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:34.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:34 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:35.582 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7/zxaroIKXRRgVicUkO0FVwM4TtcLhHq9/GZOnzlJjATu6WX1fB2Vh2qstfYqDQIYuBHg2Pee556n6YUwM8FP2LNTnd2u3lEd9WzCQLAM/J7frG3UiEB0HYiU0wSiNCH9Onkx/tl1WqphFcusaOsgU8vwa7MkOnz+e+6E4DewUv/w8D70mfajEoSOCfubbySzjKIAVPLBQ8NZXVT0IHdmb/+y4fkz0BizyoP5yepUFAl72PiuIKdLjjs0EuejdkNebBFaf/xanL9OYpVDurJuhMIRZzImS9sETseenuhiKAjMUfTBnSuBcvO022k9mb53pcU/UVhioMz3Jqt4V54n368ClJiF5GubwfOqLNtEYdJvaU9G9Kd+lS44iEZn4Zm7MheJ55KbqwcMIPeWcgRBSiOGPaMwMCjhtXsT8OHN1BjeFSaviCoXK8c2KvvQJIrx9EHjan+Av28K1O+10Cf4nXiIrK/BONZ/8axTmxS1NQ+DkkdPSfbNAyE9DzfUHMs= ceph-eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:08:35.582 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f627ece4700 1 Processor -- start 2024-04-26T21:08:35.582 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f627ece4700 1 -- start start 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f627ece4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f62781090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f627ece4700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62781096b0 con 0x7f6278108cc0 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f627ca80700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f62781090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f627ca80700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f62781090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43162/0 (socket says 172.21.15.129:43162) 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f627ca80700 1 -- 172.21.15.129:0/1035907655 learned_addr learned my addr 172.21.15.129:0/1035907655 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f627ca80700 1 -- 172.21.15.129:0/1035907655 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6278109ec0 con 0x7f6278108cc0 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f627ca80700 1 --2- 172.21.15.129:0/1035907655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f62781090e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f626c0098a0 tx=0x7f626c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bd688bff413c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.784+0000 7f62777fe700 1 -- 172.21.15.129:0/1035907655 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f626c004160 con 0x7f6278108cc0 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.785+0000 7f62777fe700 1 -- 172.21.15.129:0/1035907655 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f626c004300 con 0x7f6278108cc0 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.785+0000 7f62777fe700 1 -- 172.21.15.129:0/1035907655 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f626c011af0 con 0x7f6278108cc0 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.785+0000 7f627ece4700 1 -- 172.21.15.129:0/1035907655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 msgr2=0x7f62781090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.785+0000 7f627ece4700 1 --2- 172.21.15.129:0/1035907655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f62781090e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f626c0098a0 tx=0x7f626c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.583 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.785+0000 7f627ece4700 1 -- 172.21.15.129:0/1035907655 shutdown_connections 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.785+0000 7f627ece4700 1 --2- 172.21.15.129:0/1035907655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f62781090e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.785+0000 7f627ece4700 1 -- 172.21.15.129:0/1035907655 >> 172.21.15.129:0/1035907655 conn(0x7f6278104260 msgr2=0x7f62781066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.785+0000 7f627ece4700 1 -- 172.21.15.129:0/1035907655 shutdown_connections 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.785+0000 7f627ece4700 1 -- 172.21.15.129:0/1035907655 wait complete. 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ece4700 1 Processor -- start 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ece4700 1 -- start start 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ece4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f627819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ece4700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f627810d270 con 0x7f6278108cc0 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ca80700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f627819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ca80700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f627819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43176/0 (socket says 172.21.15.129:43176) 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ca80700 1 -- 172.21.15.129:0/3955883521 learned_addr learned my addr 172.21.15.129:0/3955883521 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ca80700 1 -- 172.21.15.129:0/3955883521 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627819d200 con 0x7f6278108cc0 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ca80700 1 --2- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f627819ccc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f626c009fb0 tx=0x7f626c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f6275ffb700 1 -- 172.21.15.129:0/3955883521 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f626c0036a0 con 0x7f6278108cc0 2024-04-26T21:08:35.584 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f6275ffb700 1 -- 172.21.15.129:0/3955883521 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f626c01a430 con 0x7f6278108cc0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.786+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627819d490 con 0x7f6278108cc0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.787+0000 7f6275ffb700 1 -- 172.21.15.129:0/3955883521 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f626c019550 con 0x7f6278108cc0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.787+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62781a0180 con 0x7f6278108cc0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.787+0000 7f6275ffb700 1 -- 172.21.15.129:0/3955883521 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f626c019850 con 0x7f6278108cc0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.788+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6278064a80 con 0x7f6278108cc0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.788+0000 7f6275ffb700 1 --2- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f6268039560 0x7f626803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.788+0000 7f6275ffb700 1 -- 172.21.15.129:0/3955883521 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f626c04da70 con 0x7f6278108cc0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.788+0000 7f6277fff700 1 --2- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f6268039560 0x7f626803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.788+0000 7f6277fff700 1 --2- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f6268039560 0x7f626803ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6264006fd0 tx=0x7f6264006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.793+0000 7f6275ffb700 1 -- 172.21.15.129:0/3955883521 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f626c023c90 con 0x7f6278108cc0 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.993+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f6278105500 con 0x7f6268039560 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.994+0000 7f6275ffb700 1 -- 172.21.15.129:0/3955883521 <== mgr.14120 v2:172.21.15.129:6800/948723446 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f6278105500 con 0x7f6268039560 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f6268039560 msgr2=0x7f626803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:35.585 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 --2- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f6268039560 0x7f626803ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6264006fd0 tx=0x7f6264006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 msgr2=0x7f627819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 --2- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f627819ccc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f626c009fb0 tx=0x7f626c011940 comp rx=0 tx=0).stop 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 shutdown_connections 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 --2- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f6268039560 0x7f626803ba20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 --2- 172.21.15.129:0/3955883521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6278108cc0 0x7f627819ccc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 >> 172.21.15.129:0/3955883521 conn(0x7f6278104260 msgr2=0x7f6278104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 shutdown_connections 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:34.996+0000 7f627ece4700 1 -- 172.21.15.129:0/3955883521 wait complete. 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T21:08:35.586 INFO:teuthology.orchestra.run.smithi129.stdout:Adding host smithi129... 2024-04-26T21:08:36.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:35 smithi129 ceph-mon[29237]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:37.064 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:36 smithi129 ceph-mon[29237]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "addr": "172.21.15.129", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:37.065 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:36 smithi129 ceph-mon[29237]: Deploying cephadm binary to smithi129 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout Added host 'smithi129' with addr '172.21.15.129' 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.010+0000 7fbb163b3700 1 Processor -- start 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.010+0000 7fbb163b3700 1 -- start start 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.010+0000 7fbb163b3700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb10107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.010+0000 7fbb163b3700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10108570 con 0x7fbb10107b80 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.011+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb10107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.011+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb10107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43180/0 (socket says 172.21.15.129:43180) 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.011+0000 7fbb0ffff700 1 -- 172.21.15.129:0/577713591 learned_addr learned my addr 172.21.15.129:0/577713591 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.011+0000 7fbb0ffff700 1 -- 172.21.15.129:0/577713591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb10108d80 con 0x7fbb10107b80 2024-04-26T21:08:40.933 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.011+0000 7fbb0ffff700 1 --2- 172.21.15.129:0/577713591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb10107fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000bb40 tx=0x7fbb0000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c312b2c574e6162d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.011+0000 7fbb0effd700 1 -- 172.21.15.129:0/577713591 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb0000dbe0 con 0x7fbb10107b80 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.011+0000 7fbb0effd700 1 -- 172.21.15.129:0/577713591 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbb000044c0 con 0x7fbb10107b80 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.011+0000 7fbb0effd700 1 -- 172.21.15.129:0/577713591 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb00010470 con 0x7fbb10107b80 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.012+0000 7fbb163b3700 1 -- 172.21.15.129:0/577713591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 msgr2=0x7fbb10107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.012+0000 7fbb163b3700 1 --2- 172.21.15.129:0/577713591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb10107fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000bb40 tx=0x7fbb0000be50 comp rx=0 tx=0).stop 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.012+0000 7fbb163b3700 1 -- 172.21.15.129:0/577713591 shutdown_connections 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.012+0000 7fbb163b3700 1 --2- 172.21.15.129:0/577713591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb10107fa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.012+0000 7fbb163b3700 1 -- 172.21.15.129:0/577713591 >> 172.21.15.129:0/577713591 conn(0x7fbb10103140 msgr2=0x7fbb10105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.012+0000 7fbb163b3700 1 -- 172.21.15.129:0/577713591 shutdown_connections 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.012+0000 7fbb163b3700 1 -- 172.21.15.129:0/577713591 wait complete. 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.013+0000 7fbb163b3700 1 Processor -- start 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.013+0000 7fbb163b3700 1 -- start start 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.013+0000 7fbb163b3700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb1019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.934 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.013+0000 7fbb163b3700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1007bfd0 con 0x7fbb10107b80 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.013+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb1019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.013+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb1019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43186/0 (socket says 172.21.15.129:43186) 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.013+0000 7fbb0ffff700 1 -- 172.21.15.129:0/3238301030 learned_addr learned my addr 172.21.15.129:0/3238301030 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.013+0000 7fbb0ffff700 1 -- 172.21.15.129:0/3238301030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1019af20 con 0x7fbb10107b80 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.013+0000 7fbb0ffff700 1 --2- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb1019a9e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000bfd0 tx=0x7fbb00003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.014+0000 7fbb0d7fa700 1 -- 172.21.15.129:0/3238301030 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb00027070 con 0x7fbb10107b80 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.014+0000 7fbb0d7fa700 1 -- 172.21.15.129:0/3238301030 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbb00010c00 con 0x7fbb10107b80 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.014+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1019b1b0 con 0x7fbb10107b80 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.014+0000 7fbb0d7fa700 1 -- 172.21.15.129:0/3238301030 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb00022c10 con 0x7fbb10107b80 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.014+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1019de50 con 0x7fbb10107b80 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.014+0000 7fbb0d7fa700 1 -- 172.21.15.129:0/3238301030 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbb0002a070 con 0x7fbb10107b80 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.014+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb10064a80 con 0x7fbb10107b80 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.015+0000 7fbb0d7fa700 1 --2- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fbaf80395b0 0x7fbaf803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.935 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.015+0000 7fbb0d7fa700 1 -- 172.21.15.129:0/3238301030 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb00022e00 con 0x7fbb10107b80 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.015+0000 7fbb0f7fe700 1 --2- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fbaf80395b0 0x7fbaf803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.015+0000 7fbb0f7fe700 1 --2- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fbaf80395b0 0x7fbaf803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbb04006fd0 tx=0x7fbb04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.020+0000 7fbb0d7fa700 1 -- 172.21.15.129:0/3238301030 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbb00015070 con 0x7fbb10107b80 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:36.227+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi129", "addr": "172.21.15.129", "target": ["mon-mgr", ""]}) -- 0x7fbb10063860 con 0x7fbaf80395b0 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.365+0000 7fbb0d7fa700 1 -- 172.21.15.129:0/3238301030 <== mgr.14120 v2:172.21.15.129:6800/948723446 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbb10063860 con 0x7fbaf80395b0 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fbaf80395b0 msgr2=0x7fbaf803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 --2- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fbaf80395b0 0x7fbaf803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbb04006fd0 tx=0x7fbb04006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 msgr2=0x7fbb1019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 --2- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb1019a9e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000bfd0 tx=0x7fbb00003ff0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 shutdown_connections 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 --2- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fbaf80395b0 0x7fbaf803ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 --2- 172.21.15.129:0/3238301030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb10107b80 0x7fbb1019a9e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.936 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 >> 172.21.15.129:0/3238301030 conn(0x7fbb10103140 msgr2=0x7fbb10191000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:40.937 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 shutdown_connections 2024-04-26T21:08:40.937 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:40.367+0000 7fbb163b3700 1 -- 172.21.15.129:0/3238301030 wait complete. 2024-04-26T21:08:40.937 INFO:teuthology.orchestra.run.smithi129.stdout:Deploying unmanaged mon service... 2024-04-26T21:08:41.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:41 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:41.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:41 smithi129 ceph-mon[29237]: Added host smithi129 2024-04-26T21:08:41.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:41 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:42.275 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T21:08:42.275 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.404+0000 7fd987ee2700 1 Processor -- start 2024-04-26T21:08:42.275 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.404+0000 7fd987ee2700 1 -- start start 2024-04-26T21:08:42.275 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.404+0000 7fd987ee2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.275 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.404+0000 7fd987ee2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd980108570 con 0x7fd980107b80 2024-04-26T21:08:42.275 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.405+0000 7fd985c7e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.275 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.405+0000 7fd985c7e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43198/0 (socket says 172.21.15.129:43198) 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.405+0000 7fd985c7e700 1 -- 172.21.15.129:0/1771093870 learned_addr learned my addr 172.21.15.129:0/1771093870 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.405+0000 7fd985c7e700 1 -- 172.21.15.129:0/1771093870 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd980108d80 con 0x7fd980107b80 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.405+0000 7fd985c7e700 1 --2- 172.21.15.129:0/1771093870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980107fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0098a0 tx=0x7fd97c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5616e1b49a0a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.405+0000 7fd984c7c700 1 -- 172.21.15.129:0/1771093870 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd97c004160 con 0x7fd980107b80 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.405+0000 7fd984c7c700 1 -- 172.21.15.129:0/1771093870 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd97c004300 con 0x7fd980107b80 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.405+0000 7fd984c7c700 1 -- 172.21.15.129:0/1771093870 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd97c011af0 con 0x7fd980107b80 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.406+0000 7fd987ee2700 1 -- 172.21.15.129:0/1771093870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 msgr2=0x7fd980107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.406+0000 7fd987ee2700 1 --2- 172.21.15.129:0/1771093870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980107fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0098a0 tx=0x7fd97c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.406+0000 7fd987ee2700 1 -- 172.21.15.129:0/1771093870 shutdown_connections 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.406+0000 7fd987ee2700 1 --2- 172.21.15.129:0/1771093870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980107fa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.406+0000 7fd987ee2700 1 -- 172.21.15.129:0/1771093870 >> 172.21.15.129:0/1771093870 conn(0x7fd980103140 msgr2=0x7fd980105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:42.276 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.406+0000 7fd987ee2700 1 -- 172.21.15.129:0/1771093870 shutdown_connections 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.406+0000 7fd987ee2700 1 -- 172.21.15.129:0/1771093870 wait complete. 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.406+0000 7fd987ee2700 1 Processor -- start 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.406+0000 7fd987ee2700 1 -- start start 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd987ee2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980192cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd987ee2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010c130 con 0x7fd980107b80 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd985c7e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980192cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd985c7e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980192cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43214/0 (socket says 172.21.15.129:43214) 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd985c7e700 1 -- 172.21.15.129:0/1685259996 learned_addr learned my addr 172.21.15.129:0/1685259996 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd985c7e700 1 -- 172.21.15.129:0/1685259996 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd980193230 con 0x7fd980107b80 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd985c7e700 1 --2- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980192cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd97c003f40 tx=0x7fd97c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd976ffd700 1 -- 172.21.15.129:0/1685259996 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd97c0038b0 con 0x7fd980107b80 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd980191420 con 0x7fd980107b80 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd976ffd700 1 -- 172.21.15.129:0/1685259996 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd97c003a50 con 0x7fd980107b80 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd976ffd700 1 -- 172.21.15.129:0/1685259996 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd97c01a5c0 con 0x7fd980107b80 2024-04-26T21:08:42.277 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.407+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd980191960 con 0x7fd980107b80 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.408+0000 7fd976ffd700 1 -- 172.21.15.129:0/1685259996 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd97c024430 con 0x7fd980107b80 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.408+0000 7fd976ffd700 1 --2- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fd96c039560 0x7fd96c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.408+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd980064a80 con 0x7fd980107b80 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.408+0000 7fd976ffd700 1 -- 172.21.15.129:0/1685259996 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd97c04dad0 con 0x7fd980107b80 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.408+0000 7fd98547d700 1 --2- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fd96c039560 0x7fd96c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.409+0000 7fd98547d700 1 --2- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fd96c039560 0x7fd96c03ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd970006fd0 tx=0x7fd970006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.413+0000 7fd976ffd700 1 -- 172.21.15.129:0/1685259996 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd97c02d910 con 0x7fd980107b80 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.628+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd9801922e0 con 0x7fd96c039560 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.635+0000 7fd976ffd700 1 -- 172.21.15.129:0/1685259996 <== mgr.14120 v2:172.21.15.129:6800/948723446 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd9801922e0 con 0x7fd96c039560 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fd96c039560 msgr2=0x7fd96c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 --2- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fd96c039560 0x7fd96c03ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd970006fd0 tx=0x7fd970006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 msgr2=0x7fd980192cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.278 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 --2- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980192cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd97c003f40 tx=0x7fd97c011890 comp rx=0 tx=0).stop 2024-04-26T21:08:42.279 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 shutdown_connections 2024-04-26T21:08:42.279 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 --2- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fd96c039560 0x7fd96c03ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.279 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 --2- 172.21.15.129:0/1685259996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd980107b80 0x7fd980192cf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.279 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 >> 172.21.15.129:0/1685259996 conn(0x7fd980103140 msgr2=0x7fd980103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:42.279 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 shutdown_connections 2024-04-26T21:08:42.279 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:41.637+0000 7fd987ee2700 1 -- 172.21.15.129:0/1685259996 wait complete. 2024-04-26T21:08:42.279 INFO:teuthology.orchestra.run.smithi129.stdout:Deploying unmanaged mgr service... 2024-04-26T21:08:42.920 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:42 smithi129 ceph-mon[29237]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:42.920 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:42 smithi129 ceph-mon[29237]: Saving service mon spec with placement count:5 2024-04-26T21:08:42.920 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:42 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:44.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:43 smithi129 ceph-mon[29237]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:44.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:43 smithi129 ceph-mon[29237]: Saving service mgr spec with placement count:2 2024-04-26T21:08:44.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:43 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:44.356 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T21:08:44.356 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.705+0000 7fba127c2700 1 Processor -- start 2024-04-26T21:08:44.356 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.705+0000 7fba127c2700 1 -- start start 2024-04-26T21:08:44.356 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.705+0000 7fba127c2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.356 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.705+0000 7fba127c2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c1085b0 con 0x7fba0c107bc0 2024-04-26T21:08:44.356 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.705+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.356 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.705+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c107fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45136/0 (socket says 172.21.15.129:45136) 2024-04-26T21:08:44.356 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.705+0000 7fba0bfff700 1 -- 172.21.15.129:0/1178085495 learned_addr learned my addr 172.21.15.129:0/1178085495 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.706+0000 7fba0bfff700 1 -- 172.21.15.129:0/1178085495 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c108dc0 con 0x7fba0c107bc0 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.706+0000 7fba0bfff700 1 --2- 172.21.15.129:0/1178085495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c107fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0098a0 tx=0x7fb9fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd10bce33781d363 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.706+0000 7fba0affd700 1 -- 172.21.15.129:0/1178085495 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9fc004160 con 0x7fba0c107bc0 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.706+0000 7fba0affd700 1 -- 172.21.15.129:0/1178085495 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb9fc004300 con 0x7fba0c107bc0 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.706+0000 7fba0affd700 1 -- 172.21.15.129:0/1178085495 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9fc011af0 con 0x7fba0c107bc0 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.707+0000 7fba127c2700 1 -- 172.21.15.129:0/1178085495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 msgr2=0x7fba0c107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.707+0000 7fba127c2700 1 --2- 172.21.15.129:0/1178085495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c107fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0098a0 tx=0x7fb9fc009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.707+0000 7fba127c2700 1 -- 172.21.15.129:0/1178085495 shutdown_connections 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.707+0000 7fba127c2700 1 --2- 172.21.15.129:0/1178085495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c107fe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.707+0000 7fba127c2700 1 -- 172.21.15.129:0/1178085495 >> 172.21.15.129:0/1178085495 conn(0x7fba0c103160 msgr2=0x7fba0c1055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.707+0000 7fba127c2700 1 -- 172.21.15.129:0/1178085495 shutdown_connections 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.707+0000 7fba127c2700 1 -- 172.21.15.129:0/1178085495 wait complete. 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.707+0000 7fba127c2700 1 Processor -- start 2024-04-26T21:08:44.357 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.707+0000 7fba127c2700 1 -- start start 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba127c2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba127c2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c10c170 con 0x7fba0c107bc0 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c19bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45144/0 (socket says 172.21.15.129:45144) 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba0bfff700 1 -- 172.21.15.129:0/1002267104 learned_addr learned my addr 172.21.15.129:0/1002267104 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba0bfff700 1 -- 172.21.15.129:0/1002267104 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c19c070 con 0x7fba0c107bc0 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba0bfff700 1 --2- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c19bb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000c00 tx=0x7fb9fc011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba097fa700 1 -- 172.21.15.129:0/1002267104 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9fc003900 con 0x7fba0c107bc0 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba097fa700 1 -- 172.21.15.129:0/1002267104 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb9fc003aa0 con 0x7fba0c107bc0 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0c19c300 con 0x7fba0c107bc0 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba097fa700 1 -- 172.21.15.129:0/1002267104 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9fc01c780 con 0x7fba0c107bc0 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.708+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba0c07bfd0 con 0x7fba0c107bc0 2024-04-26T21:08:44.358 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.709+0000 7fba097fa700 1 -- 172.21.15.129:0/1002267104 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb9fc01b450 con 0x7fba0c107bc0 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.709+0000 7fba097fa700 1 --2- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fb9f80395b0 0x7fb9f803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.709+0000 7fba097fa700 1 -- 172.21.15.129:0/1002267104 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb9fc050280 con 0x7fba0c107bc0 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.709+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba0c064a80 con 0x7fba0c107bc0 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.710+0000 7fba0b7fe700 1 --2- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fb9f80395b0 0x7fb9f803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.710+0000 7fba0b7fe700 1 --2- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fb9f80395b0 0x7fb9f803ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4006fd0 tx=0x7fb9f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.715+0000 7fba097fa700 1 -- 172.21.15.129:0/1002267104 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb9fc01a3d0 con 0x7fba0c107bc0 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.933+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fba0c104380 con 0x7fb9f80395b0 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.940+0000 7fba097fa700 1 -- 172.21.15.129:0/1002267104 <== mgr.14120 v2:172.21.15.129:6800/948723446 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fba0c104380 con 0x7fb9f80395b0 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fb9f80395b0 msgr2=0x7fb9f803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 --2- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fb9f80395b0 0x7fb9f803ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4006fd0 tx=0x7fb9f4006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 msgr2=0x7fba0c19bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 --2- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c19bb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000c00 tx=0x7fb9fc011cb0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.359 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 shutdown_connections 2024-04-26T21:08:44.360 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 --2- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fb9f80395b0 0x7fb9f803ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.360 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 --2- 172.21.15.129:0/1002267104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0c107bc0 0x7fba0c19bb30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.360 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 >> 172.21.15.129:0/1002267104 conn(0x7fba0c103160 msgr2=0x7fba0c103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.360 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 shutdown_connections 2024-04-26T21:08:44.360 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:42.942+0000 7fba127c2700 1 -- 172.21.15.129:0/1002267104 wait complete. 2024-04-26T21:08:45.329 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:44 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:45.329 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:44 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:45.563 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.819+0000 7efc6353d700 1 Processor -- start 2024-04-26T21:08:45.563 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.819+0000 7efc6353d700 1 -- start start 2024-04-26T21:08:45.563 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.819+0000 7efc6353d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c107330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.563 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.819+0000 7efc6353d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c107900 con 0x7efc5c106f10 2024-04-26T21:08:45.563 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.820+0000 7efc612d9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c107330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.563 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.820+0000 7efc612d9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c107330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45160/0 (socket says 172.21.15.129:45160) 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.820+0000 7efc612d9700 1 -- 172.21.15.129:0/3237568723 learned_addr learned my addr 172.21.15.129:0/3237568723 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.820+0000 7efc612d9700 1 -- 172.21.15.129:0/3237568723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c107a80 con 0x7efc5c106f10 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.820+0000 7efc612d9700 1 --2- 172.21.15.129:0/3237568723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c107330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098a0 tx=0x7efc4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2526ae2d88504b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.820+0000 7efc53fff700 1 -- 172.21.15.129:0/3237568723 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc4c004160 con 0x7efc5c106f10 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.820+0000 7efc53fff700 1 -- 172.21.15.129:0/3237568723 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc4c004300 con 0x7efc5c106f10 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.820+0000 7efc53fff700 1 -- 172.21.15.129:0/3237568723 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc4c011a20 con 0x7efc5c106f10 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.821+0000 7efc6353d700 1 -- 172.21.15.129:0/3237568723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 msgr2=0x7efc5c107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.821+0000 7efc6353d700 1 --2- 172.21.15.129:0/3237568723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c107330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098a0 tx=0x7efc4c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.821+0000 7efc6353d700 1 -- 172.21.15.129:0/3237568723 shutdown_connections 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.821+0000 7efc6353d700 1 --2- 172.21.15.129:0/3237568723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c107330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.821+0000 7efc6353d700 1 -- 172.21.15.129:0/3237568723 >> 172.21.15.129:0/3237568723 conn(0x7efc5c1028d0 msgr2=0x7efc5c104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.821+0000 7efc6353d700 1 -- 172.21.15.129:0/3237568723 shutdown_connections 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.821+0000 7efc6353d700 1 -- 172.21.15.129:0/3237568723 wait complete. 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.822+0000 7efc6353d700 1 Processor -- start 2024-04-26T21:08:45.564 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.822+0000 7efc6353d700 1 -- start start 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.822+0000 7efc6353d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c109810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.822+0000 7efc6353d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c10d1c0 con 0x7efc5c106f10 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.822+0000 7efc612d9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c109810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.822+0000 7efc612d9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c109810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45166/0 (socket says 172.21.15.129:45166) 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.822+0000 7efc612d9700 1 -- 172.21.15.129:0/1726923290 learned_addr learned my addr 172.21.15.129:0/1726923290 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.822+0000 7efc612d9700 1 -- 172.21.15.129:0/1726923290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c109d50 con 0x7efc5c106f10 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.822+0000 7efc612d9700 1 --2- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c109810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0116e0 tx=0x7efc4c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.823+0000 7efc527fc700 1 -- 172.21.15.129:0/1726923290 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc4c0039e0 con 0x7efc5c106f10 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.823+0000 7efc527fc700 1 -- 172.21.15.129:0/1726923290 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc4c01b8f0 con 0x7efc5c106f10 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.823+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c107f40 con 0x7efc5c106f10 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.823+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c108480 con 0x7efc5c106f10 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.823+0000 7efc527fc700 1 -- 172.21.15.129:0/1726923290 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc4c023760 con 0x7efc5c106f10 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.823+0000 7efc527fc700 1 -- 172.21.15.129:0/1726923290 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7efc4c023a00 con 0x7efc5c106f10 2024-04-26T21:08:45.565 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.824+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc40005360 con 0x7efc5c106f10 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.824+0000 7efc527fc700 1 --2- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7efc480395b0 0x7efc4803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.824+0000 7efc527fc700 1 -- 172.21.15.129:0/1726923290 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efc4c04dea0 con 0x7efc5c106f10 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.824+0000 7efc60ad8700 1 --2- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7efc480395b0 0x7efc4803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.824+0000 7efc60ad8700 1 --2- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7efc480395b0 0x7efc4803ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc58006fd0 tx=0x7efc58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:44.829+0000 7efc527fc700 1 -- 172.21.15.129:0/1726923290 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efc4c01f030 con 0x7efc5c106f10 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.035+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7efc40005650 con 0x7efc5c106f10 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.038+0000 7efc527fc700 1 -- 172.21.15.129:0/1726923290 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7efc4c01bd50 con 0x7efc5c106f10 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.040+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7efc480395b0 msgr2=0x7efc4803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.040+0000 7efc6353d700 1 --2- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7efc480395b0 0x7efc4803ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc58006fd0 tx=0x7efc58006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.040+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 msgr2=0x7efc5c109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.040+0000 7efc6353d700 1 --2- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c109810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0116e0 tx=0x7efc4c011aa0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.040+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 shutdown_connections 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.040+0000 7efc6353d700 1 --2- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7efc480395b0 0x7efc4803ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.566 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.040+0000 7efc6353d700 1 --2- 172.21.15.129:0/1726923290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc5c106f10 0x7efc5c109810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.567 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.040+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 >> 172.21.15.129:0/1726923290 conn(0x7efc5c1028d0 msgr2=0x7efc5c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.567 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.041+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 shutdown_connections 2024-04-26T21:08:45.567 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:45.041+0000 7efc6353d700 1 -- 172.21.15.129:0/1726923290 wait complete. 2024-04-26T21:08:46.337 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:46 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1726923290' entity='client.admin' 2024-04-26T21:08:47.273 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:47 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1914220933' entity='client.admin' 2024-04-26T21:08:47.274 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:47 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:47.282 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.025+0000 7fa969588700 1 Processor -- start 2024-04-26T21:08:47.282 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.025+0000 7fa969588700 1 -- start start 2024-04-26T21:08:47.282 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa969588700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa964109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.282 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa969588700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa964109710 con 0x7fa964108d20 2024-04-26T21:08:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa962ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa964109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa962ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa964109140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45168/0 (socket says 172.21.15.129:45168) 2024-04-26T21:08:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa962ffd700 1 -- 172.21.15.129:0/4005451684 learned_addr learned my addr 172.21.15.129:0/4005451684 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa962ffd700 1 -- 172.21.15.129:0/4005451684 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa964109e80 con 0x7fa964108d20 2024-04-26T21:08:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa962ffd700 1 --2- 172.21.15.129:0/4005451684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa964109140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb3ea4270e533a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa961ffb700 1 -- 172.21.15.129:0/4005451684 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa954004160 con 0x7fa964108d20 2024-04-26T21:08:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa961ffb700 1 -- 172.21.15.129:0/4005451684 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa954004300 con 0x7fa964108d20 2024-04-26T21:08:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.026+0000 7fa961ffb700 1 -- 172.21.15.129:0/4005451684 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa954011a20 con 0x7fa964108d20 2024-04-26T21:08:47.283 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.027+0000 7fa969588700 1 -- 172.21.15.129:0/4005451684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 msgr2=0x7fa964109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.027+0000 7fa969588700 1 --2- 172.21.15.129:0/4005451684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa964109140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa954009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.027+0000 7fa969588700 1 -- 172.21.15.129:0/4005451684 shutdown_connections 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.027+0000 7fa969588700 1 --2- 172.21.15.129:0/4005451684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa964109140 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.027+0000 7fa969588700 1 -- 172.21.15.129:0/4005451684 >> 172.21.15.129:0/4005451684 conn(0x7fa9641042e0 msgr2=0x7fa964106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.027+0000 7fa969588700 1 -- 172.21.15.129:0/4005451684 shutdown_connections 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.027+0000 7fa969588700 1 -- 172.21.15.129:0/4005451684 wait complete. 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.028+0000 7fa969588700 1 Processor -- start 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.028+0000 7fa969588700 1 -- start start 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.028+0000 7fa969588700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa96419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.028+0000 7fa969588700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa96410d230 con 0x7fa964108d20 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.028+0000 7fa962ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa96419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.028+0000 7fa962ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa96419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45176/0 (socket says 172.21.15.129:45176) 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.028+0000 7fa962ffd700 1 -- 172.21.15.129:0/1914220933 learned_addr learned my addr 172.21.15.129:0/1914220933 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.029+0000 7fa962ffd700 1 -- 172.21.15.129:0/1914220933 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96419d0f0 con 0x7fa964108d20 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.029+0000 7fa962ffd700 1 --2- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa96419cbb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa9540116e0 tx=0x7fa954011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.029+0000 7fa94bfff700 1 -- 172.21.15.129:0/1914220933 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9540039e0 con 0x7fa964108d20 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.029+0000 7fa94bfff700 1 -- 172.21.15.129:0/1914220933 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa95401b8f0 con 0x7fa964108d20 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.029+0000 7fa94bfff700 1 -- 172.21.15.129:0/1914220933 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa954023760 con 0x7fa964108d20 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.029+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96419d380 con 0x7fa964108d20 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.029+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9641a0070 con 0x7fa964108d20 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.030+0000 7fa94bfff700 1 -- 172.21.15.129:0/1914220933 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa954023920 con 0x7fa964108d20 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.030+0000 7fa94bfff700 1 --2- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fa94c0395b0 0x7fa94c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.030+0000 7fa94bfff700 1 -- 172.21.15.129:0/1914220933 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa95404edc0 con 0x7fa964108d20 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.030+0000 7fa9627fc700 1 --2- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fa94c0395b0 0x7fa94c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.030+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa964064a80 con 0x7fa964108d20 2024-04-26T21:08:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.030+0000 7fa9627fc700 1 --2- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fa94c0395b0 0x7fa94c03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa958006fd0 tx=0x7fa958006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.036+0000 7fa94bfff700 1 -- 172.21.15.129:0/1914220933 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa95401f030 con 0x7fa964108d20 2024-04-26T21:08:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.237+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa9641a0240 con 0x7fa964108d20 2024-04-26T21:08:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.240+0000 7fa94bfff700 1 -- 172.21.15.129:0/1914220933 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa954018e30 con 0x7fa964108d20 2024-04-26T21:08:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.244+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fa94c0395b0 msgr2=0x7fa94c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.244+0000 7fa969588700 1 --2- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fa94c0395b0 0x7fa94c03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa958006fd0 tx=0x7fa958006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.244+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 msgr2=0x7fa96419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.244+0000 7fa969588700 1 --2- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa96419cbb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa9540116e0 tx=0x7fa954011aa0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.244+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 shutdown_connections 2024-04-26T21:08:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.244+0000 7fa969588700 1 --2- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7fa94c0395b0 0x7fa94c03ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.244+0000 7fa969588700 1 --2- 172.21.15.129:0/1914220933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa964108d20 0x7fa96419cbb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.244+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 >> 172.21.15.129:0/1914220933 conn(0x7fa9641042e0 msgr2=0x7fa964104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.245+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 shutdown_connections 2024-04-26T21:08:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:46.245+0000 7fa969588700 1 -- 172.21.15.129:0/1914220933 wait complete. 2024-04-26T21:08:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:Enabling the dashboard module... 2024-04-26T21:08:48.261 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:48 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1678575901' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T21:08:48.262 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:48 smithi129 ceph-mon[29237]: from='mgr.14120 172.21.15.129:0/2131505744' entity='mgr.x' 2024-04-26T21:08:48.631 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: ignoring --setuser ceph since I am not root 2024-04-26T21:08:48.631 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: ignoring --setgroup ceph since I am not root 2024-04-26T21:08:48.631 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:48.440+0000 7f2812758200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:08:48.965 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:48.637+0000 7f2812758200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:48.966 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:48.740+0000 7f2812758200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:08:48.979 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.723+0000 7f5ea4fde700 1 Processor -- start 2024-04-26T21:08:48.979 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.723+0000 7f5ea4fde700 1 -- start start 2024-04-26T21:08:48.980 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.724+0000 7f5ea4fde700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea01090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.980 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.724+0000 7f5ea4fde700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01096b0 con 0x7f5ea0108cc0 2024-04-26T21:08:48.980 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.724+0000 7f5e9e59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea01090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.980 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.724+0000 7f5e9e59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea01090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45178/0 (socket says 172.21.15.129:45178) 2024-04-26T21:08:48.980 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.724+0000 7f5e9e59c700 1 -- 172.21.15.129:0/2464786437 learned_addr learned my addr 172.21.15.129:0/2464786437 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:48.980 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.724+0000 7f5e9e59c700 1 -- 172.21.15.129:0/2464786437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea0109e70 con 0x7f5ea0108cc0 2024-04-26T21:08:48.980 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.724+0000 7f5e9e59c700 1 --2- 172.21.15.129:0/2464786437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea01090e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009e80 tx=0x7f5e9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15299af76853c967 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.981 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5e9d59a700 1 -- 172.21.15.129:0/2464786437 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e98017070 con 0x7f5ea0108cc0 2024-04-26T21:08:48.981 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5e9d59a700 1 -- 172.21.15.129:0/2464786437 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9800b8c0 con 0x7f5ea0108cc0 2024-04-26T21:08:48.981 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5e9d59a700 1 -- 172.21.15.129:0/2464786437 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e98004490 con 0x7f5ea0108cc0 2024-04-26T21:08:48.981 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5ea4fde700 1 -- 172.21.15.129:0/2464786437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 msgr2=0x7f5ea01090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.981 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5ea4fde700 1 --2- 172.21.15.129:0/2464786437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea01090e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009e80 tx=0x7f5e9800b330 comp rx=0 tx=0).stop 2024-04-26T21:08:48.981 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5ea4fde700 1 -- 172.21.15.129:0/2464786437 shutdown_connections 2024-04-26T21:08:48.981 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5ea4fde700 1 --2- 172.21.15.129:0/2464786437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea01090e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.982 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5ea4fde700 1 -- 172.21.15.129:0/2464786437 >> 172.21.15.129:0/2464786437 conn(0x7f5ea0104280 msgr2=0x7f5ea01066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:48.982 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5ea4fde700 1 -- 172.21.15.129:0/2464786437 shutdown_connections 2024-04-26T21:08:48.982 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.725+0000 7f5ea4fde700 1 -- 172.21.15.129:0/2464786437 wait complete. 2024-04-26T21:08:48.982 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.726+0000 7f5ea4fde700 1 Processor -- start 2024-04-26T21:08:48.982 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.726+0000 7f5ea4fde700 1 -- start start 2024-04-26T21:08:48.982 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.726+0000 7f5ea4fde700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.982 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.726+0000 7f5ea4fde700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea010d220 con 0x7f5ea0108cc0 2024-04-26T21:08:48.983 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.726+0000 7f5e9e59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.983 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.726+0000 7f5e9e59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45180/0 (socket says 172.21.15.129:45180) 2024-04-26T21:08:48.983 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.726+0000 7f5e9e59c700 1 -- 172.21.15.129:0/1678575901 learned_addr learned my addr 172.21.15.129:0/1678575901 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:48.983 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.727+0000 7f5e9e59c700 1 -- 172.21.15.129:0/1678575901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea019d140 con 0x7f5ea0108cc0 2024-04-26T21:08:48.983 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.727+0000 7f5e9e59c700 1 --2- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea019cc00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800b7b0 tx=0x7f5e9800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.983 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.727+0000 7f5e977fe700 1 -- 172.21.15.129:0/1678575901 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e98020020 con 0x7f5ea0108cc0 2024-04-26T21:08:48.983 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.727+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea019d3d0 con 0x7f5ea0108cc0 2024-04-26T21:08:48.984 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.727+0000 7f5e977fe700 1 -- 172.21.15.129:0/1678575901 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e98003b50 con 0x7f5ea0108cc0 2024-04-26T21:08:48.984 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.727+0000 7f5e977fe700 1 -- 172.21.15.129:0/1678575901 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e98017040 con 0x7f5ea0108cc0 2024-04-26T21:08:48.984 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.727+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea01a00c0 con 0x7f5ea0108cc0 2024-04-26T21:08:48.984 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.728+0000 7f5e977fe700 1 -- 172.21.15.129:0/1678575901 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5e98023020 con 0x7f5ea0108cc0 2024-04-26T21:08:48.984 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.728+0000 7f5e977fe700 1 --2- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e8c0395b0 0x7f5e8c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.984 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.728+0000 7f5e977fe700 1 -- 172.21.15.129:0/1678575901 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5e9800f050 con 0x7f5ea0108cc0 2024-04-26T21:08:48.984 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.728+0000 7f5e9dd9b700 1 --2- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e8c0395b0 0x7f5e8c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.985 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.728+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea0064a80 con 0x7f5ea0108cc0 2024-04-26T21:08:48.985 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.729+0000 7f5e9dd9b700 1 --2- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e8c0395b0 0x7f5e8c03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5e88006fd0 tx=0x7f5e88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.985 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.733+0000 7f5e977fe700 1 -- 172.21.15.129:0/1678575901 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5e9802a020 con 0x7f5ea0108cc0 2024-04-26T21:08:48.985 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:47.998+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5ea01a0520 con 0x7f5ea0108cc0 2024-04-26T21:08:48.985 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.244+0000 7f5e977fe700 1 -- 172.21.15.129:0/1678575901 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5e98003d00 con 0x7f5ea0108cc0 2024-04-26T21:08:48.985 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.246+0000 7f5e977fe700 1 -- 172.21.15.129:0/1678575901 <== mon.0 v2:172.21.15.129:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5e98025400 con 0x7f5ea0108cc0 2024-04-26T21:08:48.986 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.247+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e8c0395b0 msgr2=0x7f5e8c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.986 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.247+0000 7f5ea4fde700 1 --2- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e8c0395b0 0x7f5e8c03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5e88006fd0 tx=0x7f5e88006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:48.986 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.247+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 msgr2=0x7f5ea019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.986 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.247+0000 7f5ea4fde700 1 --2- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea019cc00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800b7b0 tx=0x7f5e9800bd30 comp rx=0 tx=0).stop 2024-04-26T21:08:48.986 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.248+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 shutdown_connections 2024-04-26T21:08:48.986 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.248+0000 7f5ea4fde700 1 --2- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e8c0395b0 0x7f5e8c03ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.987 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.248+0000 7f5ea4fde700 1 --2- 172.21.15.129:0/1678575901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0108cc0 0x7f5ea019cc00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.987 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.248+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 >> 172.21.15.129:0/1678575901 conn(0x7f5ea0104280 msgr2=0x7f5ea0104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:48.987 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.248+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 shutdown_connections 2024-04-26T21:08:48.987 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:48.248+0000 7f5ea4fde700 1 -- 172.21.15.129:0/1678575901 wait complete. 2024-04-26T21:08:49.217 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:49 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:49.054+0000 7f2812758200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:08:49.217 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:49 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:49.139+0000 7f2812758200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:08:49.591 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:49 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1678575901' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T21:08:49.591 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:49 smithi129 ceph-mon[29237]: mgrmap e9: x(active, since 19s) 2024-04-26T21:08:50.373 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:50 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2796653818' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:08:50.721 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:50 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:50.372+0000 7f2812758200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:08:50.722 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:50 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:50.546+0000 7f2812758200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:51.016 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:50 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:50.720+0000 7f2812758200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:08:51.016 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:51.015+0000 7f2812758200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:08:51.295 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:51.110+0000 7f2812758200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.440+0000 7f5e9225a700 1 Processor -- start 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.440+0000 7f5e9225a700 1 -- start start 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.441+0000 7f5e9225a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c107360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.441+0000 7f5e9225a700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c107930 con 0x7f5e8c106f40 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.441+0000 7f5e8b7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c107360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.441+0000 7f5e8b7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c107360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45210/0 (socket says 172.21.15.129:45210) 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.441+0000 7f5e8b7fe700 1 -- 172.21.15.129:0/2422384717 learned_addr learned my addr 172.21.15.129:0/2422384717 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.441+0000 7f5e8b7fe700 1 -- 172.21.15.129:0/2422384717 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c109ed0 con 0x7f5e8c106f40 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.441+0000 7f5e8b7fe700 1 --2- 172.21.15.129:0/2422384717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c107360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5e800098a0 tx=0x7f5e80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3cc38eb5a3c98a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.441+0000 7f5e8a7fc700 1 -- 172.21.15.129:0/2422384717 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e80004160 con 0x7f5e8c106f40 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.442+0000 7f5e8a7fc700 1 -- 172.21.15.129:0/2422384717 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e80004300 con 0x7f5e8c106f40 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.442+0000 7f5e8a7fc700 1 -- 172.21.15.129:0/2422384717 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e80011a20 con 0x7f5e8c106f40 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.442+0000 7f5e9225a700 1 -- 172.21.15.129:0/2422384717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 msgr2=0x7f5e8c107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.442+0000 7f5e9225a700 1 --2- 172.21.15.129:0/2422384717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c107360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5e800098a0 tx=0x7f5e80009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.442+0000 7f5e9225a700 1 -- 172.21.15.129:0/2422384717 shutdown_connections 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.442+0000 7f5e9225a700 1 --2- 172.21.15.129:0/2422384717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c107360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.442+0000 7f5e9225a700 1 -- 172.21.15.129:0/2422384717 >> 172.21.15.129:0/2422384717 conn(0x7f5e8c1028d0 msgr2=0x7f5e8c104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.443+0000 7f5e9225a700 1 -- 172.21.15.129:0/2422384717 shutdown_connections 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.443+0000 7f5e9225a700 1 -- 172.21.15.129:0/2422384717 wait complete. 2024-04-26T21:08:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.443+0000 7f5e9225a700 1 Processor -- start 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.443+0000 7f5e9225a700 1 -- start start 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.443+0000 7f5e9225a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.443+0000 7f5e9225a700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c10d280 con 0x7f5e8c106f40 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e8b7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e8b7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45216/0 (socket says 172.21.15.129:45216) 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e8b7fe700 1 -- 172.21.15.129:0/2796653818 learned_addr learned my addr 172.21.15.129:0/2796653818 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e8b7fe700 1 -- 172.21.15.129:0/2796653818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c19d100 con 0x7f5e8c106f40 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e8b7fe700 1 --2- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c19cbc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e800116e0 tx=0x7f5e80011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e88ff9700 1 -- 172.21.15.129:0/2796653818 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e800039e0 con 0x7f5e8c106f40 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e88ff9700 1 -- 172.21.15.129:0/2796653818 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e8001b8f0 con 0x7f5e8c106f40 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c19d390 con 0x7f5e8c106f40 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e88ff9700 1 -- 172.21.15.129:0/2796653818 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e80023760 con 0x7f5e8c106f40 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.444+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1a0080 con 0x7f5e8c106f40 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.445+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c064a80 con 0x7f5e8c106f40 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.445+0000 7f5e88ff9700 1 -- 172.21.15.129:0/2796653818 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5e8001b440 con 0x7f5e8c106f40 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.445+0000 7f5e88ff9700 1 --2- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e74039610 0x7f5e7403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.445+0000 7f5e88ff9700 1 -- 172.21.15.129:0/2796653818 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5e8004df30 con 0x7f5e8c106f40 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.445+0000 7f5e8affd700 1 -- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e74039610 msgr2=0x7f5e7403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/948723446 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.446+0000 7f5e8affd700 1 --2- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e74039610 0x7f5e7403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.450+0000 7f5e88ff9700 1 -- 172.21.15.129:0/2796653818 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5e8001f030 con 0x7f5e8c106f40 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.647+0000 7f5e8affd700 1 -- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e74039610 msgr2=0x7f5e7403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/948723446 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.647+0000 7f5e8affd700 1 --2- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e74039610 0x7f5e7403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.762+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5e8c1a0630 con 0x7f5e8c106f40 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.762+0000 7f5e88ff9700 1 -- 172.21.15.129:0/2796653818 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f5e8002c3f0 con 0x7f5e8c106f40 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.764+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e74039610 msgr2=0x7f5e7403bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.764+0000 7f5e9225a700 1 --2- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e74039610 0x7f5e7403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.764+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 msgr2=0x7f5e8c19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.764+0000 7f5e9225a700 1 --2- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c19cbc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e800116e0 tx=0x7f5e80011aa0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.764+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 shutdown_connections 2024-04-26T21:08:51.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.764+0000 7f5e9225a700 1 --2- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7f5e74039610 0x7f5e7403bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.764+0000 7f5e9225a700 1 --2- 172.21.15.129:0/2796653818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e8c106f40 0x7f5e8c19cbc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.764+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 >> 172.21.15.129:0/2796653818 conn(0x7f5e8c1028d0 msgr2=0x7f5e8c1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:51.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.765+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 shutdown_connections 2024-04-26T21:08:51.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:49.765+0000 7f5e9225a700 1 -- 172.21.15.129:0/2796653818 wait complete. 2024-04-26T21:08:51.308 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for the mgr to restart... 2024-04-26T21:08:51.308 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mgr epoch 9... 2024-04-26T21:08:51.565 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:51.319+0000 7f2812758200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:08:51.565 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:51.517+0000 7f2812758200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:52.372 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:52.132+0000 7f2812758200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:08:52.650 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:52.553+0000 7f2812758200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:08:52.650 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:52.649+0000 7f2812758200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:08:52.970 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:52.852+0000 7f2812758200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:08:53.466 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:53.373+0000 7f2812758200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:08:53.720 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:53.465+0000 7f2812758200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:08:53.720 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:53.706+0000 7f2812758200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:08:54.220 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:53.841+0000 7f2812758200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:08:54.220 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:53.937+0000 7f2812758200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:08:54.220 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:54 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:54.051+0000 7f2812758200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:54.720 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:54 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:54.355+0000 7f2812758200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:08:55.172 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:54 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:54.956+0000 7f2812758200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:08:55.470 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:08:55.171+0000 7f2812758200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:08:55.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: Active manager daemon x restarted 2024-04-26T21:08:55.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: Activating manager daemon x 2024-04-26T21:08:55.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T21:08:55.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: mgrmap e10: x(active, starting, since 0.00351493s) 2024-04-26T21:08:55.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:08:55.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:08:55.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:08:55.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:08:55.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:08:55.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: Manager daemon x is now available 2024-04-26T21:08:55.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:55.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:08:56.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:08:56.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:56 smithi129 ceph-mon[29237]: mgrmap e11: x(active, since 1.00633s) 2024-04-26T21:08:57.165 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:08:57.165 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T21:08:57.165 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:08:57.165 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:08:57.165 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.764+0000 7ff60154d700 1 Processor -- start 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.764+0000 7ff60154d700 1 -- start start 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.764+0000 7ff60154d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc07d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.764+0000 7ff60154d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc07d8f0 con 0x7ff5fc07eec0 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.765+0000 7ff5faffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc07d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.765+0000 7ff5faffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc07d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45228/0 (socket says 172.21.15.129:45228) 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.765+0000 7ff5faffd700 1 -- 172.21.15.129:0/4248139760 learned_addr learned my addr 172.21.15.129:0/4248139760 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.765+0000 7ff5faffd700 1 -- 172.21.15.129:0/4248139760 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc07da70 con 0x7ff5fc07eec0 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.765+0000 7ff5faffd700 1 --2- 172.21.15.129:0/4248139760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc07d320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0098a0 tx=0x7ff5ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c0576e0259e914 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.765+0000 7ff5f9ffb700 1 -- 172.21.15.129:0/4248139760 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec004160 con 0x7ff5fc07eec0 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.765+0000 7ff5f9ffb700 1 -- 172.21.15.129:0/4248139760 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5ec004300 con 0x7ff5fc07eec0 2024-04-26T21:08:57.166 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.765+0000 7ff5f9ffb700 1 -- 172.21.15.129:0/4248139760 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec011a20 con 0x7ff5fc07eec0 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.766+0000 7ff60154d700 1 -- 172.21.15.129:0/4248139760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 msgr2=0x7ff5fc07d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.766+0000 7ff60154d700 1 --2- 172.21.15.129:0/4248139760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc07d320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0098a0 tx=0x7ff5ec009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.766+0000 7ff60154d700 1 -- 172.21.15.129:0/4248139760 shutdown_connections 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.766+0000 7ff60154d700 1 --2- 172.21.15.129:0/4248139760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc07d320 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.766+0000 7ff60154d700 1 -- 172.21.15.129:0/4248139760 >> 172.21.15.129:0/4248139760 conn(0x7ff5fc104240 msgr2=0x7ff5fc106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.766+0000 7ff60154d700 1 -- 172.21.15.129:0/4248139760 shutdown_connections 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.766+0000 7ff60154d700 1 -- 172.21.15.129:0/4248139760 wait complete. 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.767+0000 7ff60154d700 1 Processor -- start 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.767+0000 7ff60154d700 1 -- start start 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.767+0000 7ff60154d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.767+0000 7ff60154d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc10f280 con 0x7ff5fc07eec0 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.767+0000 7ff5faffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.767+0000 7ff5faffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45234/0 (socket says 172.21.15.129:45234) 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.767+0000 7ff5faffd700 1 -- 172.21.15.129:0/2232284676 learned_addr learned my addr 172.21.15.129:0/2232284676 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.768+0000 7ff5faffd700 1 -- 172.21.15.129:0/2232284676 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19f330 con 0x7ff5fc07eec0 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.768+0000 7ff5faffd700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc19edf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0119f0 tx=0x7ff5ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.768+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec0039e0 con 0x7ff5fc07eec0 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.768+0000 7ff60154d700 1 -- 172.21.15.129:0/2232284676 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc19f5c0 con 0x7ff5fc07eec0 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.768+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5ec01b8f0 con 0x7ff5fc07eec0 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.768+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec0237a0 con 0x7ff5fc07eec0 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.768+0000 7ff60154d700 1 -- 172.21.15.129:0/2232284676 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a22b0 con 0x7ff5fc07eec0 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.769+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff5ec02c3f0 con 0x7ff5fc07eec0 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.769+0000 7ff5e3fff700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 0x7ff5e403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.769+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 --> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff5e403c180 con 0x7ff5e40395b0 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.769+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5ec04dec0 con 0x7ff5fc07eec0 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.769+0000 7ff5fa7fc700 1 -- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 msgr2=0x7ff5e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/948723446 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.769+0000 7ff5fa7fc700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 0x7ff5e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.970+0000 7ff5fa7fc700 1 -- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 msgr2=0x7ff5e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/948723446 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:51.970+0000 7ff5fa7fc700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 0x7ff5e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:52.371+0000 7ff5fa7fc700 1 -- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 msgr2=0x7ff5e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/948723446 2024-04-26T21:08:57.169 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:52.371+0000 7ff5fa7fc700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 0x7ff5e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:53.172+0000 7ff5fa7fc700 1 -- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 msgr2=0x7ff5e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/948723446 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:53.172+0000 7ff5fa7fc700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 0x7ff5e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:54.773+0000 7ff5fa7fc700 1 -- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 msgr2=0x7ff5e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/948723446 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:54.773+0000 7ff5fa7fc700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 0x7ff5e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:55.175+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7ff5ec02bc70 con 0x7ff5fc07eec0 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:55.175+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 msgr2=0x7ff5e403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:55.175+0000 7ff5e3fff700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 0x7ff5e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.177+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7ff5ec04fb10 con 0x7ff5fc07eec0 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.177+0000 7ff5e3fff700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff5e403cfe0 0x7ff5e403f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.177+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff5ec0169b0 con 0x7ff5e403cfe0 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.177+0000 7ff5fa7fc700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff5e403cfe0 0x7ff5e403f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.178+0000 7ff5fa7fc700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff5e403cfe0 0x7ff5e403f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0003a10 tx=0x7ff5f00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.179+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff5ec0169b0 con 0x7ff5e403cfe0 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.187+0000 7ff60154d700 1 -- 172.21.15.129:0/2232284676 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff5fc06a520 con 0x7ff5e403cfe0 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.187+0000 7ff5e3fff700 1 -- 172.21.15.129:0/2232284676 <== mgr.14152 v2:172.21.15.129:6800/3856000307 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff5fc06a520 con 0x7ff5e403cfe0 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.187+0000 7ff60154d700 1 -- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff5e403cfe0 msgr2=0x7ff5e403f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.187+0000 7ff60154d700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff5e403cfe0 0x7ff5e403f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0003a10 tx=0x7ff5f00092b0 comp rx=0 tx=0).stop 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.187+0000 7ff60154d700 1 -- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 msgr2=0x7ff5fc19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.187+0000 7ff60154d700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc19edf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0119f0 tx=0x7ff5ec011ad0 comp rx=0 tx=0).stop 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.187+0000 7ff60154d700 1 -- 172.21.15.129:0/2232284676 shutdown_connections 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.187+0000 7ff60154d700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5fc07eec0 0x7ff5fc19edf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.188+0000 7ff60154d700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff5e403cfe0 0x7ff5e403f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.188+0000 7ff60154d700 1 --2- 172.21.15.129:0/2232284676 >> [v2:172.21.15.129:6800/948723446,v1:172.21.15.129:6801/948723446] conn(0x7ff5e40395b0 0x7ff5e403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.188+0000 7ff60154d700 1 -- 172.21.15.129:0/2232284676 >> 172.21.15.129:0/2232284676 conn(0x7ff5fc104240 msgr2=0x7ff5fc104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:57.172 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.188+0000 7ff60154d700 1 -- 172.21.15.129:0/2232284676 shutdown_connections 2024-04-26T21:08:57.172 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:56.188+0000 7ff60154d700 1 -- 172.21.15.129:0/2232284676 wait complete. 2024-04-26T21:08:57.172 INFO:teuthology.orchestra.run.smithi129.stdout:mgr epoch 9 is available 2024-04-26T21:08:57.172 INFO:teuthology.orchestra.run.smithi129.stdout:Generating a dashboard self-signed certificate... 2024-04-26T21:08:57.457 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:57 smithi129 ceph-mon[29237]: [26/Apr/2024:21:08:55] ENGINE Bus STARTING 2024-04-26T21:08:57.457 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:57 smithi129 ceph-mon[29237]: [26/Apr/2024:21:08:55] ENGINE Serving on https://172.21.15.129:7150 2024-04-26T21:08:57.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:57 smithi129 ceph-mon[29237]: [26/Apr/2024:21:08:55] ENGINE Serving on http://172.21.15.129:8765 2024-04-26T21:08:57.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:57 smithi129 ceph-mon[29237]: [26/Apr/2024:21:08:55] ENGINE Bus STARTED 2024-04-26T21:08:57.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:57 smithi129 ceph-mon[29237]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:08:57.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:57 smithi129 ceph-mon[29237]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:08:57.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:08:58.696 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:58 smithi129 ceph-mon[29237]: mgrmap e12: x(active, since 2s) 2024-04-26T21:08:58.696 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:08:58.696 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:58 smithi129 ceph-mon[29237]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:58.696 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:08:58.696 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:08:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.685+0000 7f32d2b93700 1 Processor -- start 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.685+0000 7f32d2b93700 1 -- start start 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.686+0000 7f32d2b93700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc10bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.686+0000 7f32d2b93700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc07ce70 con 0x7f32cc109730 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.686+0000 7f32d092f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc10bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.686+0000 7f32d092f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc10bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48646/0 (socket says 172.21.15.129:48646) 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.686+0000 7f32d092f700 1 -- 172.21.15.129:0/1686898956 learned_addr learned my addr 172.21.15.129:0/1686898956 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.686+0000 7f32d092f700 1 -- 172.21.15.129:0/1686898956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc10c060 con 0x7f32cc109730 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.686+0000 7f32d092f700 1 --2- 172.21.15.129:0/1686898956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc10bb20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f32c00098a0 tx=0x7f32c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585414450eae8c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32cb7fe700 1 -- 172.21.15.129:0/1686898956 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32c0004160 con 0x7f32cc109730 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32cb7fe700 1 -- 172.21.15.129:0/1686898956 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c0004300 con 0x7f32cc109730 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32cb7fe700 1 -- 172.21.15.129:0/1686898956 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32c0011a20 con 0x7f32cc109730 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32d2b93700 1 -- 172.21.15.129:0/1686898956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 msgr2=0x7f32cc10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32d2b93700 1 --2- 172.21.15.129:0/1686898956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc10bb20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f32c00098a0 tx=0x7f32c0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32d2b93700 1 -- 172.21.15.129:0/1686898956 shutdown_connections 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32d2b93700 1 --2- 172.21.15.129:0/1686898956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc10bb20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32d2b93700 1 -- 172.21.15.129:0/1686898956 >> 172.21.15.129:0/1686898956 conn(0x7f32cc103140 msgr2=0x7f32cc105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32d2b93700 1 -- 172.21.15.129:0/1686898956 shutdown_connections 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.687+0000 7f32d2b93700 1 -- 172.21.15.129:0/1686898956 wait complete. 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.688+0000 7f32d2b93700 1 Processor -- start 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.688+0000 7f32d2b93700 1 -- start start 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.688+0000 7f32d2b93700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.688+0000 7f32d2b93700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10d060 con 0x7f32cc109730 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.688+0000 7f32d092f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.688+0000 7f32d092f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48648/0 (socket says 172.21.15.129:48648) 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.688+0000 7f32d092f700 1 -- 172.21.15.129:0/1294485877 learned_addr learned my addr 172.21.15.129:0/1294485877 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.689+0000 7f32d092f700 1 -- 172.21.15.129:0/1294485877 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc19d0c0 con 0x7f32cc109730 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.689+0000 7f32d092f700 1 --2- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc19cb80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f32c00116e0 tx=0x7f32c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.689+0000 7f32c9ffb700 1 -- 172.21.15.129:0/1294485877 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32c0003a60 con 0x7f32cc109730 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.689+0000 7f32c9ffb700 1 -- 172.21.15.129:0/1294485877 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c001b8f0 con 0x7f32cc109730 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.689+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc19d350 con 0x7f32cc109730 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.689+0000 7f32c9ffb700 1 -- 172.21.15.129:0/1294485877 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32c0019860 con 0x7f32cc109730 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.689+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a0040 con 0x7f32cc109730 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.690+0000 7f32c9ffb700 1 -- 172.21.15.129:0/1294485877 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f32c001b440 con 0x7f32cc109730 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.690+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc064a80 con 0x7f32cc109730 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.690+0000 7f32c9ffb700 1 --2- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f32bc039610 0x7f32bc03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.690+0000 7f32c9ffb700 1 -- 172.21.15.129:0/1294485877 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f32c004ddd0 con 0x7f32cc109730 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.690+0000 7f32cbfff700 1 --2- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f32bc039610 0x7f32bc03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.691+0000 7f32cbfff700 1 --2- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f32bc039610 0x7f32bc03bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32b8006fd0 tx=0x7f32b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.695+0000 7f32c9ffb700 1 -- 172.21.15.129:0/1294485877 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f32c0051030 con 0x7f32cc109730 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:57.908+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f32cc104340 con 0x7f32bc039610 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.068+0000 7f32c9ffb700 1 -- 172.21.15.129:0/1294485877 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f32cc104340 con 0x7f32bc039610 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.070+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f32bc039610 msgr2=0x7f32bc03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.070+0000 7f32d2b93700 1 --2- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f32bc039610 0x7f32bc03bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32b8006fd0 tx=0x7f32b8006e40 comp rx=0 tx=0).stop 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.070+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 msgr2=0x7f32cc19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.070+0000 7f32d2b93700 1 --2- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc19cb80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f32c00116e0 tx=0x7f32c0011aa0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.070+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 shutdown_connections 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.070+0000 7f32d2b93700 1 --2- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f32bc039610 0x7f32bc03bad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.070+0000 7f32d2b93700 1 --2- 172.21.15.129:0/1294485877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32cc109730 0x7f32cc19cb80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.070+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 >> 172.21.15.129:0/1294485877 conn(0x7f32cc103140 msgr2=0x7f32cc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.071+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 shutdown_connections 2024-04-26T21:08:58.708 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:58.071+0000 7f32d2b93700 1 -- 172.21.15.129:0/1294485877 wait complete. 2024-04-26T21:08:58.708 INFO:teuthology.orchestra.run.smithi129.stdout:Creating initial admin user... 2024-04-26T21:09:00.893 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:00 smithi129 ceph-mon[29237]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:00.893 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:00.901 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$DCkzyXx2HZv5DDdemSRLGuLKg66QWHqHYQPBbRTphMqUeSLbbrcAW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714165739, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T21:09:00.901 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.162+0000 7fac7143d700 1 Processor -- start 2024-04-26T21:09:00.901 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.162+0000 7fac7143d700 1 -- start start 2024-04-26T21:09:00.901 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.162+0000 7fac7143d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c10bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.901 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.162+0000 7fac7143d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c07ce70 con 0x7fac6c109770 2024-04-26T21:09:00.901 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.162+0000 7fac6affd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c10bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.901 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.162+0000 7fac6affd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c10bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48658/0 (socket says 172.21.15.129:48658) 2024-04-26T21:09:00.901 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.162+0000 7fac6affd700 1 -- 172.21.15.129:0/2707064805 learned_addr learned my addr 172.21.15.129:0/2707064805 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:00.901 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.163+0000 7fac6affd700 1 -- 172.21.15.129:0/2707064805 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c10c0a0 con 0x7fac6c109770 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.163+0000 7fac6affd700 1 --2- 172.21.15.129:0/2707064805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c10bb60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0098a0 tx=0x7fac5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30a7f8c4050db05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.163+0000 7fac69ffb700 1 -- 172.21.15.129:0/2707064805 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac5c004160 con 0x7fac6c109770 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.163+0000 7fac69ffb700 1 -- 172.21.15.129:0/2707064805 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5c004300 con 0x7fac6c109770 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.163+0000 7fac69ffb700 1 -- 172.21.15.129:0/2707064805 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac5c011a20 con 0x7fac6c109770 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.164+0000 7fac7143d700 1 -- 172.21.15.129:0/2707064805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 msgr2=0x7fac6c10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.164+0000 7fac7143d700 1 --2- 172.21.15.129:0/2707064805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c10bb60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0098a0 tx=0x7fac5c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.164+0000 7fac7143d700 1 -- 172.21.15.129:0/2707064805 shutdown_connections 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.164+0000 7fac7143d700 1 --2- 172.21.15.129:0/2707064805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c10bb60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.164+0000 7fac7143d700 1 -- 172.21.15.129:0/2707064805 >> 172.21.15.129:0/2707064805 conn(0x7fac6c103180 msgr2=0x7fac6c1055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.164+0000 7fac7143d700 1 -- 172.21.15.129:0/2707064805 shutdown_connections 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.164+0000 7fac7143d700 1 -- 172.21.15.129:0/2707064805 wait complete. 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.165+0000 7fac7143d700 1 Processor -- start 2024-04-26T21:09:00.902 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.165+0000 7fac7143d700 1 -- start start 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.165+0000 7fac7143d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c1a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.165+0000 7fac7143d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c10d0a0 con 0x7fac6c109770 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.165+0000 7fac6affd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c1a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.165+0000 7fac6affd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c1a31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48664/0 (socket says 172.21.15.129:48664) 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.165+0000 7fac6affd700 1 -- 172.21.15.129:0/3449686144 learned_addr learned my addr 172.21.15.129:0/3449686144 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.165+0000 7fac6affd700 1 -- 172.21.15.129:0/3449686144 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c1a3700 con 0x7fac6c109770 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.166+0000 7fac6affd700 1 --2- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c1a31c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac5c000c00 tx=0x7fac5c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.166+0000 7fac53fff700 1 -- 172.21.15.129:0/3449686144 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac5c003aa0 con 0x7fac6c109770 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.166+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c1a3990 con 0x7fac6c109770 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.166+0000 7fac53fff700 1 -- 172.21.15.129:0/3449686144 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5c01b8f0 con 0x7fac6c109770 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.166+0000 7fac53fff700 1 -- 172.21.15.129:0/3449686144 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac5c023870 con 0x7fac6c109770 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.166+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c1a6680 con 0x7fac6c109770 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.167+0000 7fac53fff700 1 -- 172.21.15.129:0/3449686144 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fac5c023b70 con 0x7fac6c109770 2024-04-26T21:09:00.903 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.167+0000 7fac53fff700 1 --2- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac54039600 0x7fac5403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.167+0000 7fac53fff700 1 -- 172.21.15.129:0/3449686144 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fac5c04ef10 con 0x7fac6c109770 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.167+0000 7fac6a7fc700 1 --2- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac54039600 0x7fac5403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.167+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c064a80 con 0x7fac6c109770 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.167+0000 7fac6a7fc700 1 --2- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac54039600 0x7fac5403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac60006fd0 tx=0x7fac60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.173+0000 7fac53fff700 1 -- 172.21.15.129:0/3449686144 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fac5c01f030 con 0x7fac6c109770 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.396+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fac6c104270 con 0x7fac54039600 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.621+0000 7fac53fff700 1 -- 172.21.15.129:0/3449686144 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fac6c104270 con 0x7fac54039600 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac54039600 msgr2=0x7fac5403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 --2- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac54039600 0x7fac5403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac60006fd0 tx=0x7fac60006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 msgr2=0x7fac6c1a31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 --2- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c1a31c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac5c000c00 tx=0x7fac5c011aa0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 shutdown_connections 2024-04-26T21:09:00.904 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 --2- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac54039600 0x7fac5403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.905 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 --2- 172.21.15.129:0/3449686144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac6c109770 0x7fac6c1a31c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.905 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 >> 172.21.15.129:0/3449686144 conn(0x7fac6c103180 msgr2=0x7fac6c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:00.905 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 shutdown_connections 2024-04-26T21:09:00.905 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:08:59.623+0000 7fac7143d700 1 -- 172.21.15.129:0/3449686144 wait complete. 2024-04-26T21:09:00.905 INFO:teuthology.orchestra.run.smithi129.stdout:Fetching dashboard port number... 2024-04-26T21:09:01.842 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:01 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1712697217' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T21:09:02.100 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T21:09:02.100 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.318+0000 7fb2e0b09700 1 Processor -- start 2024-04-26T21:09:02.100 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.318+0000 7fb2e0b09700 1 -- start start 2024-04-26T21:09:02.100 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.318+0000 7fb2e0b09700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc10bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.318+0000 7fb2e0b09700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc07ce30 con 0x7fb2dc109780 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.318+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc10bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.318+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc10bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48680/0 (socket says 172.21.15.129:48680) 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.318+0000 7fb2da59c700 1 -- 172.21.15.129:0/3328704161 learned_addr learned my addr 172.21.15.129:0/3328704161 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.319+0000 7fb2da59c700 1 -- 172.21.15.129:0/3328704161 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc10c0b0 con 0x7fb2dc109780 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.319+0000 7fb2da59c700 1 --2- 172.21.15.129:0/3328704161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc10bb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00098a0 tx=0x7fb2d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c646044927a89d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.319+0000 7fb2d959a700 1 -- 172.21.15.129:0/3328704161 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2d0004160 con 0x7fb2dc109780 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.319+0000 7fb2d959a700 1 -- 172.21.15.129:0/3328704161 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d0004300 con 0x7fb2dc109780 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.319+0000 7fb2d959a700 1 -- 172.21.15.129:0/3328704161 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2d0011a20 con 0x7fb2dc109780 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.320+0000 7fb2e0b09700 1 -- 172.21.15.129:0/3328704161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 msgr2=0x7fb2dc10bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.320+0000 7fb2e0b09700 1 --2- 172.21.15.129:0/3328704161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc10bb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00098a0 tx=0x7fb2d0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.320+0000 7fb2e0b09700 1 -- 172.21.15.129:0/3328704161 shutdown_connections 2024-04-26T21:09:02.101 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.320+0000 7fb2e0b09700 1 --2- 172.21.15.129:0/3328704161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc10bb70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.320+0000 7fb2e0b09700 1 -- 172.21.15.129:0/3328704161 >> 172.21.15.129:0/3328704161 conn(0x7fb2dc1030f0 msgr2=0x7fb2dc105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.320+0000 7fb2e0b09700 1 -- 172.21.15.129:0/3328704161 shutdown_connections 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.320+0000 7fb2e0b09700 1 -- 172.21.15.129:0/3328704161 wait complete. 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.320+0000 7fb2e0b09700 1 Processor -- start 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.320+0000 7fb2e0b09700 1 -- start start 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2e0b09700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2e0b09700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc10d050 con 0x7fb2dc109780 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48690/0 (socket says 172.21.15.129:48690) 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2da59c700 1 -- 172.21.15.129:0/1712697217 learned_addr learned my addr 172.21.15.129:0/1712697217 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2da59c700 1 -- 172.21.15.129:0/1712697217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc19d0f0 con 0x7fb2dc109780 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2da59c700 1 --2- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc19cbb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00116e0 tx=0x7fb2d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:02.102 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2cb7fe700 1 -- 172.21.15.129:0/1712697217 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2d0003a60 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2cb7fe700 1 -- 172.21.15.129:0/1712697217 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d001b8f0 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc19d380 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2cb7fe700 1 -- 172.21.15.129:0/1712697217 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2d0019860 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.321+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc1a0070 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.322+0000 7fb2cb7fe700 1 -- 172.21.15.129:0/1712697217 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fb2d0019b10 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.322+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2dc064a80 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.322+0000 7fb2cb7fe700 1 --2- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2c4039610 0x7fb2c403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.323+0000 7fb2cb7fe700 1 -- 172.21.15.129:0/1712697217 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb2d004dee0 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.323+0000 7fb2d9d9b700 1 --2- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2c4039610 0x7fb2c403bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.323+0000 7fb2d9d9b700 1 --2- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2c4039610 0x7fb2c403bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc006fd0 tx=0x7fb2cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.328+0000 7fb2cb7fe700 1 -- 172.21.15.129:0/1712697217 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb2d001f030 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.537+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fb2dc1a0620 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.537+0000 7fb2cb7fe700 1 -- 172.21.15.129:0/1712697217 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fb2d0029260 con 0x7fb2dc109780 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.539+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2c4039610 msgr2=0x7fb2c403bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:02.103 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.539+0000 7fb2e0b09700 1 --2- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2c4039610 0x7fb2c403bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc006fd0 tx=0x7fb2cc006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:02.104 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.540+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 msgr2=0x7fb2dc19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:02.104 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.540+0000 7fb2e0b09700 1 --2- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc19cbb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00116e0 tx=0x7fb2d0011aa0 comp rx=0 tx=0).stop 2024-04-26T21:09:02.104 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.540+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 shutdown_connections 2024-04-26T21:09:02.104 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.540+0000 7fb2e0b09700 1 --2- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2c4039610 0x7fb2c403bad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:02.104 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.540+0000 7fb2e0b09700 1 --2- 172.21.15.129:0/1712697217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2dc109780 0x7fb2dc19cbb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:02.104 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.540+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 >> 172.21.15.129:0/1712697217 conn(0x7fb2dc1030f0 msgr2=0x7fb2dc103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:02.104 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.540+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 shutdown_connections 2024-04-26T21:09:02.104 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:01.540+0000 7fb2e0b09700 1 -- 172.21.15.129:0/1712697217 wait complete. 2024-04-26T21:09:02.107 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:09:02.107 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout disabled 2024-04-26T21:09:02.115 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:09:02.142 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-04-26T21:09:02.142 INFO:teuthology.orchestra.run.smithi129.stdout:firewalld.service is not enabled 2024-04-26T21:09:02.142 INFO:teuthology.orchestra.run.smithi129.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T21:09:02.143 INFO:teuthology.orchestra.run.smithi129.stdout:Ceph Dashboard is now available at: 2024-04-26T21:09:02.143 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:02.143 INFO:teuthology.orchestra.run.smithi129.stdout: URL: https://smithi129:8443/ 2024-04-26T21:09:02.143 INFO:teuthology.orchestra.run.smithi129.stdout: User: admin 2024-04-26T21:09:02.143 INFO:teuthology.orchestra.run.smithi129.stdout: Password: ml7x10pcnp 2024-04-26T21:09:02.143 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:02.144 INFO:teuthology.orchestra.run.smithi129.stdout:Saving cluster configuration to /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config directory 2024-04-26T21:09:02.144 INFO:teuthology.orchestra.run.smithi129.stdout:Enabling autotune for osd_memory_target 2024-04-26T21:09:02.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:02 smithi129 ceph-mon[29237]: mgrmap e13: x(active, since 6s) 2024-04-26T21:09:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.553+0000 7fcf522c7700 1 Processor -- start 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.553+0000 7fcf522c7700 1 -- start start 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.553+0000 7fcf522c7700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c1073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.553+0000 7fcf522c7700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c107980 con 0x7fcf4c106f90 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.553+0000 7fcf4bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c1073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.553+0000 7fcf4bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c1073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52898/0 (socket says 172.21.15.129:52898) 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.553+0000 7fcf4bfff700 1 -- 172.21.15.129:0/991692268 learned_addr learned my addr 172.21.15.129:0/991692268 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.554+0000 7fcf4bfff700 1 -- 172.21.15.129:0/991692268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c107b00 con 0x7fcf4c106f90 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.554+0000 7fcf4bfff700 1 --2- 172.21.15.129:0/991692268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c1073b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0098a0 tx=0x7fcf3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca9cc014c7a2c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.554+0000 7fcf4affd700 1 -- 172.21.15.129:0/991692268 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf3c004160 con 0x7fcf4c106f90 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.554+0000 7fcf4affd700 1 -- 172.21.15.129:0/991692268 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c004300 con 0x7fcf4c106f90 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.554+0000 7fcf4affd700 1 -- 172.21.15.129:0/991692268 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf3c011a20 con 0x7fcf4c106f90 2024-04-26T21:09:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.555+0000 7fcf522c7700 1 -- 172.21.15.129:0/991692268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 msgr2=0x7fcf4c1073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.555+0000 7fcf522c7700 1 --2- 172.21.15.129:0/991692268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c1073b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0098a0 tx=0x7fcf3c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.555+0000 7fcf522c7700 1 -- 172.21.15.129:0/991692268 shutdown_connections 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.555+0000 7fcf522c7700 1 --2- 172.21.15.129:0/991692268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c1073b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.555+0000 7fcf522c7700 1 -- 172.21.15.129:0/991692268 >> 172.21.15.129:0/991692268 conn(0x7fcf4c102910 msgr2=0x7fcf4c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.555+0000 7fcf522c7700 1 -- 172.21.15.129:0/991692268 shutdown_connections 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.555+0000 7fcf522c7700 1 -- 172.21.15.129:0/991692268 wait complete. 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.555+0000 7fcf522c7700 1 Processor -- start 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.555+0000 7fcf522c7700 1 -- start start 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf522c7700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf522c7700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c10aea0 con 0x7fcf4c106f90 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf4bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf4bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52902/0 (socket says 172.21.15.129:52902) 2024-04-26T21:09:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf4bfff700 1 -- 172.21.15.129:0/1130923360 learned_addr learned my addr 172.21.15.129:0/1130923360 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf4bfff700 1 -- 172.21.15.129:0/1130923360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c19d070 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf4bfff700 1 --2- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c19cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0116e0 tx=0x7fcf3c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf497fa700 1 -- 172.21.15.129:0/1130923360 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf3c0039e0 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c19d360 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf497fa700 1 -- 172.21.15.129:0/1130923360 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c01b8f0 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf497fa700 1 -- 172.21.15.129:0/1130923360 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf3c023700 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.556+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c19fff0 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.557+0000 7fcf497fa700 1 -- 172.21.15.129:0/1130923360 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fcf3c023970 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.557+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c064a80 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.557+0000 7fcf497fa700 1 --2- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf34039650 0x7fcf3403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.558+0000 7fcf497fa700 1 -- 172.21.15.129:0/1130923360 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcf3c04e1c0 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.558+0000 7fcf4b7fe700 1 --2- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf34039650 0x7fcf3403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.558+0000 7fcf4b7fe700 1 --2- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf34039650 0x7fcf3403bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcf40006fd0 tx=0x7fcf40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.563+0000 7fcf497fa700 1 -- 172.21.15.129:0/1130923360 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcf3c01f030 con 0x7fcf4c106f90 2024-04-26T21:09:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.763+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fcf4c1a02e0 con 0x7fcf4c106f90 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.763+0000 7fcf497fa700 1 -- 172.21.15.129:0/1130923360 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fcf3c018c10 con 0x7fcf4c106f90 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.765+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf34039650 msgr2=0x7fcf3403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.765+0000 7fcf522c7700 1 --2- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf34039650 0x7fcf3403bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcf40006fd0 tx=0x7fcf40006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.765+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 msgr2=0x7fcf4c19cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.765+0000 7fcf522c7700 1 --2- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c19cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0116e0 tx=0x7fcf3c011aa0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.766+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 shutdown_connections 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.766+0000 7fcf522c7700 1 --2- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf34039650 0x7fcf3403bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.766+0000 7fcf522c7700 1 --2- 172.21.15.129:0/1130923360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf4c106f90 0x7fcf4c19cb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.766+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 >> 172.21.15.129:0/1130923360 conn(0x7fcf4c102910 msgr2=0x7fcf4c1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.766+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 shutdown_connections 2024-04-26T21:09:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:02.766+0000 7fcf522c7700 1 -- 172.21.15.129:0/1130923360 wait complete. 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.914+0000 7f6112f34700 1 Processor -- start 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.914+0000 7f6112f34700 1 -- start start 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.914+0000 7f6112f34700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c1090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.914+0000 7f6112f34700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1096c0 con 0x7f610c108cd0 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.914+0000 7f6110cd0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c1090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.914+0000 7f6110cd0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c1090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52916/0 (socket says 172.21.15.129:52916) 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.914+0000 7f6110cd0700 1 -- 172.21.15.129:0/4110737228 learned_addr learned my addr 172.21.15.129:0/4110737228 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.915+0000 7f6110cd0700 1 -- 172.21.15.129:0/4110737228 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c109ed0 con 0x7f610c108cd0 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.915+0000 7f6110cd0700 1 --2- 172.21.15.129:0/4110737228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c1090f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61000098a0 tx=0x7f6100009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2335705385edda0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.915+0000 7f610b7fe700 1 -- 172.21.15.129:0/4110737228 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6100004160 con 0x7f610c108cd0 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.915+0000 7f610b7fe700 1 -- 172.21.15.129:0/4110737228 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6100004300 con 0x7f610c108cd0 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.915+0000 7f610b7fe700 1 -- 172.21.15.129:0/4110737228 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6100011a20 con 0x7f610c108cd0 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 -- 172.21.15.129:0/4110737228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 msgr2=0x7f610c1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 --2- 172.21.15.129:0/4110737228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c1090f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61000098a0 tx=0x7f6100009bb0 comp rx=0 tx=0).stop 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 -- 172.21.15.129:0/4110737228 shutdown_connections 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 --2- 172.21.15.129:0/4110737228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c1090f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 -- 172.21.15.129:0/4110737228 >> 172.21.15.129:0/4110737228 conn(0x7f610c1042b0 msgr2=0x7f610c1066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 -- 172.21.15.129:0/4110737228 shutdown_connections 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 -- 172.21.15.129:0/4110737228 wait complete. 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 Processor -- start 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 -- start start 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.916+0000 7f6112f34700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f610c10d280 con 0x7f610c108cd0 2024-04-26T21:09:04.833 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6110cd0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6110cd0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52928/0 (socket says 172.21.15.129:52928) 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6110cd0700 1 -- 172.21.15.129:0/2765911550 learned_addr learned my addr 172.21.15.129:0/2765911550 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6110cd0700 1 -- 172.21.15.129:0/2765911550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c19d140 con 0x7f610c108cd0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6110cd0700 1 --2- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c19cc00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61000119f0 tx=0x7f6100011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6109ffb700 1 -- 172.21.15.129:0/2765911550 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61000039e0 con 0x7f610c108cd0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6109ffb700 1 -- 172.21.15.129:0/2765911550 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610001b8f0 con 0x7f610c108cd0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610c19d3d0 con 0x7f610c108cd0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6109ffb700 1 -- 172.21.15.129:0/2765911550 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61000237a0 con 0x7f610c108cd0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.917+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610c1a00c0 con 0x7f610c108cd0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.918+0000 7f6109ffb700 1 -- 172.21.15.129:0/2765911550 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f6100023aa0 con 0x7f610c108cd0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.918+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610c064a80 con 0x7f610c108cd0 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.918+0000 7f6109ffb700 1 --2- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60fc039650 0x7f60fc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:04.834 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.918+0000 7f6109ffb700 1 -- 172.21.15.129:0/2765911550 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f610004e120 con 0x7f610c108cd0 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.918+0000 7f610bfff700 1 --2- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60fc039650 0x7f60fc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.919+0000 7f610bfff700 1 --2- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60fc039650 0x7f60fc03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60f8006fd0 tx=0x7f60f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:03.923+0000 7f6109ffb700 1 -- 172.21.15.129:0/2765911550 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f610001f030 con 0x7f610c108cd0 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.229+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f610c1a0670 con 0x7f610c108cd0 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.232+0000 7f6109ffb700 1 -- 172.21.15.129:0/2765911550 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f6100018c10 con 0x7f610c108cd0 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60fc039650 msgr2=0x7f60fc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 --2- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60fc039650 0x7f60fc03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60f8006fd0 tx=0x7f60f8006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 msgr2=0x7f610c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 --2- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c19cc00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61000119f0 tx=0x7f6100011ad0 comp rx=0 tx=0).stop 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 shutdown_connections 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 --2- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60fc039650 0x7f60fc03bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:04.835 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 --2- 172.21.15.129:0/2765911550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f610c108cd0 0x7f610c19cc00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 >> 172.21.15.129:0/2765911550 conn(0x7f610c1042b0 msgr2=0x7f610c104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 shutdown_connections 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-04-26T21:09:04.234+0000 7f6112f34700 1 -- 172.21.15.129:0/2765911550 wait complete. 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T21:09:04.836 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:04.837 INFO:teuthology.orchestra.run.smithi129.stdout: ceph telemetry on 2024-04-26T21:09:04.837 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:04.837 INFO:teuthology.orchestra.run.smithi129.stdout:For more information see: 2024-04-26T21:09:04.837 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:04.837 INFO:teuthology.orchestra.run.smithi129.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T21:09:04.837 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:04.837 INFO:teuthology.orchestra.run.smithi129.stdout:Bootstrap complete. 2024-04-26T21:09:04.866 INFO:tasks.cephadm:Fetching config... 2024-04-26T21:09:04.866 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:09:04.866 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T21:09:04.881 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T21:09:04.881 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:09:04.881 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T21:09:04.937 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T21:09:04.937 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:09:04.937 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T21:09:05.000 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T21:09:05.000 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:09:05.001 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T21:09:05.057 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T21:09:05.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7/zxaroIKXRRgVicUkO0FVwM4TtcLhHq9/GZOnzlJjATu6WX1fB2Vh2qstfYqDQIYuBHg2Pee556n6YUwM8FP2LNTnd2u3lEd9WzCQLAM/J7frG3UiEB0HYiU0wSiNCH9Onkx/tl1WqphFcusaOsgU8vwa7MkOnz+e+6E4DewUv/w8D70mfajEoSOCfubbySzjKIAVPLBQ8NZXVT0IHdmb/+y4fkz0BizyoP5yepUFAl72PiuIKdLjjs0EuejdkNebBFaf/xanL9OYpVDurJuhMIRZzImS9sETseenuhiKAjMUfTBnSuBcvO022k9mb53pcU/UVhioMz3Jqt4V54n368ClJiF5GubwfOqLNtEYdJvaU9G9Kd+lS44iEZn4Zm7MheJ55KbqwcMIPeWcgRBSiOGPaMwMCjhtXsT8OHN1BjeFSaviCoXK8c2KvvQJIrx9EHjan+Av28K1O+10Cf4nXiIrK/BONZ/8axTmxS1NQ+DkkdPSfbNAyE9DzfUHMs= ceph-eccca2b2-0410-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:09:05.128 INFO:teuthology.orchestra.run.smithi129.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7/zxaroIKXRRgVicUkO0FVwM4TtcLhHq9/GZOnzlJjATu6WX1fB2Vh2qstfYqDQIYuBHg2Pee556n6YUwM8FP2LNTnd2u3lEd9WzCQLAM/J7frG3UiEB0HYiU0wSiNCH9Onkx/tl1WqphFcusaOsgU8vwa7MkOnz+e+6E4DewUv/w8D70mfajEoSOCfubbySzjKIAVPLBQ8NZXVT0IHdmb/+y4fkz0BizyoP5yepUFAl72PiuIKdLjjs0EuejdkNebBFaf/xanL9OYpVDurJuhMIRZzImS9sETseenuhiKAjMUfTBnSuBcvO022k9mb53pcU/UVhioMz3Jqt4V54n368ClJiF5GubwfOqLNtEYdJvaU9G9Kd+lS44iEZn4Zm7MheJ55KbqwcMIPeWcgRBSiOGPaMwMCjhtXsT8OHN1BjeFSaviCoXK8c2KvvQJIrx9EHjan+Av28K1O+10Cf4nXiIrK/BONZ/8axTmxS1NQ+DkkdPSfbNAyE9DzfUHMs= ceph-eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:05.139 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7/zxaroIKXRRgVicUkO0FVwM4TtcLhHq9/GZOnzlJjATu6WX1fB2Vh2qstfYqDQIYuBHg2Pee556n6YUwM8FP2LNTnd2u3lEd9WzCQLAM/J7frG3UiEB0HYiU0wSiNCH9Onkx/tl1WqphFcusaOsgU8vwa7MkOnz+e+6E4DewUv/w8D70mfajEoSOCfubbySzjKIAVPLBQ8NZXVT0IHdmb/+y4fkz0BizyoP5yepUFAl72PiuIKdLjjs0EuejdkNebBFaf/xanL9OYpVDurJuhMIRZzImS9sETseenuhiKAjMUfTBnSuBcvO022k9mb53pcU/UVhioMz3Jqt4V54n368ClJiF5GubwfOqLNtEYdJvaU9G9Kd+lS44iEZn4Zm7MheJ55KbqwcMIPeWcgRBSiOGPaMwMCjhtXsT8OHN1BjeFSaviCoXK8c2KvvQJIrx9EHjan+Av28K1O+10Cf4nXiIrK/BONZ/8axTmxS1NQ+DkkdPSfbNAyE9DzfUHMs= ceph-eccca2b2-0410-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:09:05.178 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7/zxaroIKXRRgVicUkO0FVwM4TtcLhHq9/GZOnzlJjATu6WX1fB2Vh2qstfYqDQIYuBHg2Pee556n6YUwM8FP2LNTnd2u3lEd9WzCQLAM/J7frG3UiEB0HYiU0wSiNCH9Onkx/tl1WqphFcusaOsgU8vwa7MkOnz+e+6E4DewUv/w8D70mfajEoSOCfubbySzjKIAVPLBQ8NZXVT0IHdmb/+y4fkz0BizyoP5yepUFAl72PiuIKdLjjs0EuejdkNebBFaf/xanL9OYpVDurJuhMIRZzImS9sETseenuhiKAjMUfTBnSuBcvO022k9mb53pcU/UVhioMz3Jqt4V54n368ClJiF5GubwfOqLNtEYdJvaU9G9Kd+lS44iEZn4Zm7MheJ55KbqwcMIPeWcgRBSiOGPaMwMCjhtXsT8OHN1BjeFSaviCoXK8c2KvvQJIrx9EHjan+Av28K1O+10Cf4nXiIrK/BONZ/8axTmxS1NQ+DkkdPSfbNAyE9DzfUHMs= ceph-eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:05.190 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7/zxaroIKXRRgVicUkO0FVwM4TtcLhHq9/GZOnzlJjATu6WX1fB2Vh2qstfYqDQIYuBHg2Pee556n6YUwM8FP2LNTnd2u3lEd9WzCQLAM/J7frG3UiEB0HYiU0wSiNCH9Onkx/tl1WqphFcusaOsgU8vwa7MkOnz+e+6E4DewUv/w8D70mfajEoSOCfubbySzjKIAVPLBQ8NZXVT0IHdmb/+y4fkz0BizyoP5yepUFAl72PiuIKdLjjs0EuejdkNebBFaf/xanL9OYpVDurJuhMIRZzImS9sETseenuhiKAjMUfTBnSuBcvO022k9mb53pcU/UVhioMz3Jqt4V54n368ClJiF5GubwfOqLNtEYdJvaU9G9Kd+lS44iEZn4Zm7MheJ55KbqwcMIPeWcgRBSiOGPaMwMCjhtXsT8OHN1BjeFSaviCoXK8c2KvvQJIrx9EHjan+Av28K1O+10Cf4nXiIrK/BONZ/8axTmxS1NQ+DkkdPSfbNAyE9DzfUHMs= ceph-eccca2b2-0410-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:09:05.228 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7/zxaroIKXRRgVicUkO0FVwM4TtcLhHq9/GZOnzlJjATu6WX1fB2Vh2qstfYqDQIYuBHg2Pee556n6YUwM8FP2LNTnd2u3lEd9WzCQLAM/J7frG3UiEB0HYiU0wSiNCH9Onkx/tl1WqphFcusaOsgU8vwa7MkOnz+e+6E4DewUv/w8D70mfajEoSOCfubbySzjKIAVPLBQ8NZXVT0IHdmb/+y4fkz0BizyoP5yepUFAl72PiuIKdLjjs0EuejdkNebBFaf/xanL9OYpVDurJuhMIRZzImS9sETseenuhiKAjMUfTBnSuBcvO022k9mb53pcU/UVhioMz3Jqt4V54n368ClJiF5GubwfOqLNtEYdJvaU9G9Kd+lS44iEZn4Zm7MheJ55KbqwcMIPeWcgRBSiOGPaMwMCjhtXsT8OHN1BjeFSaviCoXK8c2KvvQJIrx9EHjan+Av28K1O+10Cf4nXiIrK/BONZ/8axTmxS1NQ+DkkdPSfbNAyE9DzfUHMs= ceph-eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:05.240 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T21:09:05.410 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:05 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2765911550' entity='client.admin' 2024-04-26T21:09:05.479 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:08.085 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:08.085 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:08.085 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:09:08.085 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:08.085 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:08.085 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.113+0000 7f0fe0709700 1 -- 172.21.15.129:0/3435261759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 msgr2=0x7f0fd8101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.113+0000 7f0fe0709700 1 --2- 172.21.15.129:0/3435261759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 0x7f0fd8101980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b0a0 tx=0x7f0fd000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.113+0000 7f0fe0709700 1 -- 172.21.15.129:0/3435261759 shutdown_connections 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.113+0000 7f0fe0709700 1 --2- 172.21.15.129:0/3435261759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 0x7f0fd8101980 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.113+0000 7f0fe0709700 1 -- 172.21.15.129:0/3435261759 >> 172.21.15.129:0/3435261759 conn(0x7f0fd80fcb00 msgr2=0x7f0fd80fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.113+0000 7f0fe0709700 1 -- 172.21.15.129:0/3435261759 shutdown_connections 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.113+0000 7f0fe0709700 1 -- 172.21.15.129:0/3435261759 wait complete. 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.114+0000 7f0fe0709700 1 Processor -- start 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.114+0000 7f0fe0709700 1 -- start start 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.114+0000 7f0fe0709700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 0x7f0fd8197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.114+0000 7f0fe0709700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd8107430 con 0x7f0fd8101560 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.114+0000 7f0fde4a5700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 0x7f0fd8197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.114+0000 7f0fde4a5700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 0x7f0fd8197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52948/0 (socket says 172.21.15.129:52948) 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.114+0000 7f0fde4a5700 1 -- 172.21.15.129:0/2552561359 learned_addr learned my addr 172.21.15.129:0/2552561359 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.114+0000 7f0fde4a5700 1 -- 172.21.15.129:0/2552561359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd8197840 con 0x7f0fd8101560 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.114+0000 7f0fde4a5700 1 --2- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 0x7f0fd8197300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000d010 tx=0x7f0fd0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.115+0000 7f0fd77fe700 1 -- 172.21.15.129:0/2552561359 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fd0021020 con 0x7f0fd8101560 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.115+0000 7f0fd77fe700 1 -- 172.21.15.129:0/2552561359 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fd000bdd0 con 0x7f0fd8101560 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.115+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd8197ad0 con 0x7f0fd8101560 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.115+0000 7f0fd77fe700 1 -- 172.21.15.129:0/2552561359 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fd0017040 con 0x7f0fd8101560 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.115+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd819a7c0 con 0x7f0fd8101560 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.116+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd8197f90 con 0x7f0fd8101560 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.116+0000 7f0fd77fe700 1 -- 172.21.15.129:0/2552561359 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f0fd0024020 con 0x7f0fd8101560 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.116+0000 7f0fd77fe700 1 --2- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0fc0039650 0x7f0fc003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.116+0000 7f0fd77fe700 1 -- 172.21.15.129:0/2552561359 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0fd000f050 con 0x7f0fd8101560 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.116+0000 7f0fddca4700 1 --2- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0fc0039650 0x7f0fc003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.116+0000 7f0fddca4700 1 --2- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0fc0039650 0x7f0fc003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:08.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.121+0000 7f0fd77fe700 1 -- 172.21.15.129:0/2552561359 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0fd002b020 con 0x7f0fd8101560 2024-04-26T21:09:08.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.328+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f0fd802e5a0 con 0x7f0fd8101560 2024-04-26T21:09:08.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.330+0000 7f0fd77fe700 1 -- 172.21.15.129:0/2552561359 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0fd0011be0 con 0x7f0fd8101560 2024-04-26T21:09:08.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0fc0039650 msgr2=0x7f0fc003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 --2- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0fc0039650 0x7f0fc003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:08.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 msgr2=0x7f0fd8197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 --2- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 0x7f0fd8197300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000d010 tx=0x7f0fd0003ce0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 shutdown_connections 2024-04-26T21:09:08.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 --2- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0fc0039650 0x7f0fc003bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 --2- 172.21.15.129:0/2552561359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fd8101560 0x7f0fd8197300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 >> 172.21.15.129:0/2552561359 conn(0x7f0fd80fcb00 msgr2=0x7f0fd818dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:08.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 shutdown_connections 2024-04-26T21:09:08.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:08.333+0000 7f0fe0709700 1 -- 172.21.15.129:0/2552561359 wait complete. 2024-04-26T21:09:09.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:09 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2552561359' entity='client.admin' 2024-04-26T21:09:09.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:09 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:09.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:09 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:09.598 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T21:09:09.598 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T21:09:09.835 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:11.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.380+0000 7fd083911700 1 -- 172.21.15.129:0/2935769124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 msgr2=0x7fd07c101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.380+0000 7fd083911700 1 --2- 172.21.15.129:0/2935769124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 0x7fd07c101930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd07800aa20 tx=0x7fd07800ad30 comp rx=0 tx=0).stop 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.380+0000 7fd083911700 1 -- 172.21.15.129:0/2935769124 shutdown_connections 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.380+0000 7fd083911700 1 --2- 172.21.15.129:0/2935769124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 0x7fd07c101930 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.381+0000 7fd083911700 1 -- 172.21.15.129:0/2935769124 >> 172.21.15.129:0/2935769124 conn(0x7fd07c0fcad0 msgr2=0x7fd07c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.381+0000 7fd083911700 1 -- 172.21.15.129:0/2935769124 shutdown_connections 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.381+0000 7fd083911700 1 -- 172.21.15.129:0/2935769124 wait complete. 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.381+0000 7fd083911700 1 Processor -- start 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.381+0000 7fd083911700 1 -- start start 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.381+0000 7fd083911700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 0x7fd07c0fe940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.381+0000 7fd083911700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c106bd0 con 0x7fd07c101510 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.381+0000 7fd0816ad700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 0x7fd07c0fe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.382+0000 7fd0816ad700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 0x7fd07c0fe940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52978/0 (socket says 172.21.15.129:52978) 2024-04-26T21:09:11.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.382+0000 7fd0816ad700 1 -- 172.21.15.129:0/3834134234 learned_addr learned my addr 172.21.15.129:0/3834134234 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.382+0000 7fd0816ad700 1 -- 172.21.15.129:0/3834134234 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c0fee80 con 0x7fd07c101510 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.382+0000 7fd0816ad700 1 --2- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 0x7fd07c0fe940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd078004e70 tx=0x7fd078004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.382+0000 7fd0727fc700 1 -- 172.21.15.129:0/3834134234 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd078005570 con 0x7fd07c101510 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.382+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07c0fd070 con 0x7fd07c101510 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.382+0000 7fd0727fc700 1 -- 172.21.15.129:0/3834134234 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd078016920 con 0x7fd07c101510 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.382+0000 7fd0727fc700 1 -- 172.21.15.129:0/3834134234 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd07800e880 con 0x7fd07c101510 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.382+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07c0fd530 con 0x7fd07c101510 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.383+0000 7fd0727fc700 1 -- 172.21.15.129:0/3834134234 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fd07800eb40 con 0x7fd07c101510 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.383+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07c107350 con 0x7fd07c101510 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.383+0000 7fd0727fc700 1 --2- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd068039650 0x7fd06803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.383+0000 7fd0727fc700 1 -- 172.21.15.129:0/3834134234 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd07801e070 con 0x7fd07c101510 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.383+0000 7fd080eac700 1 --2- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd068039650 0x7fd06803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.384+0000 7fd080eac700 1 --2- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd068039650 0x7fd06803bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd06c006fd0 tx=0x7fd06c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.388+0000 7fd0727fc700 1 -- 172.21.15.129:0/3834134234 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd078015610 con 0x7fd07c101510 2024-04-26T21:09:11.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.590+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fd07c191940 con 0x7fd068039650 2024-04-26T21:09:11.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.593+0000 7fd0727fc700 1 -- 172.21.15.129:0/3834134234 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd07c191940 con 0x7fd068039650 2024-04-26T21:09:11.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd068039650 msgr2=0x7fd06803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 --2- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd068039650 0x7fd06803bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd06c006fd0 tx=0x7fd06c006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:11.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 msgr2=0x7fd07c0fe940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 --2- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 0x7fd07c0fe940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd078004e70 tx=0x7fd078004f50 comp rx=0 tx=0).stop 2024-04-26T21:09:11.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 shutdown_connections 2024-04-26T21:09:11.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 --2- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd068039650 0x7fd06803bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 --2- 172.21.15.129:0/3834134234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd07c101510 0x7fd07c0fe940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 >> 172.21.15.129:0/3834134234 conn(0x7fd07c0fcad0 msgr2=0x7fd07c18c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:11.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 shutdown_connections 2024-04-26T21:09:11.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:11.595+0000 7fd083911700 1 -- 172.21.15.129:0/3834134234 wait complete. 2024-04-26T21:09:12.338 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-04-26T21:09:12.339 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:09:12.339 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:09:12.357 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:09:12.357 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:09:12.415 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-04-26T21:09:12.415 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch host add smithi150 2024-04-26T21:09:12.652 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:12.881 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:12.882 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:12.882 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:12.882 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:12.882 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:12.882 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-26T21:09:12.882 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: Updating smithi129:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:12.882 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:09:12.882 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: Updating smithi129:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:09:12.882 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:12.883 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:12.883 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:14.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.135+0000 7f00f1123700 1 -- 172.21.15.129:0/3730142797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 msgr2=0x7f00ec102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.135+0000 7f00f1123700 1 --2- 172.21.15.129:0/3730142797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 0x7f00ec102370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009910 tx=0x7f00dc009c20 comp rx=0 tx=0).stop 2024-04-26T21:09:14.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.135+0000 7f00f1123700 1 -- 172.21.15.129:0/3730142797 shutdown_connections 2024-04-26T21:09:14.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.135+0000 7f00f1123700 1 --2- 172.21.15.129:0/3730142797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 0x7f00ec102370 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.135+0000 7f00f1123700 1 -- 172.21.15.129:0/3730142797 >> 172.21.15.129:0/3730142797 conn(0x7f00ec0fd860 msgr2=0x7f00ec0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.135+0000 7f00f1123700 1 -- 172.21.15.129:0/3730142797 shutdown_connections 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.135+0000 7f00f1123700 1 -- 172.21.15.129:0/3730142797 wait complete. 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.136+0000 7f00f1123700 1 Processor -- start 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.136+0000 7f00f1123700 1 -- start start 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.136+0000 7f00f1123700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 0x7f00ec1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.136+0000 7f00f1123700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec10c210 con 0x7f00ec101f90 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.136+0000 7f00ead9d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 0x7f00ec1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.136+0000 7f00ead9d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 0x7f00ec1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36124/0 (socket says 172.21.15.129:36124) 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.136+0000 7f00ead9d700 1 -- 172.21.15.129:0/3589368957 learned_addr learned my addr 172.21.15.129:0/3589368957 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:14.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.136+0000 7f00ead9d700 1 -- 172.21.15.129:0/3589368957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec06bee0 con 0x7f00ec101f90 2024-04-26T21:09:14.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.137+0000 7f00ead9d700 1 --2- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 0x7f00ec1974e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0056d0 tx=0x7f00dc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.137+0000 7f00d3fff700 1 -- 172.21.15.129:0/3589368957 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00dc026070 con 0x7f00ec101f90 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.137+0000 7f00d3fff700 1 -- 172.21.15.129:0/3589368957 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc00bba0 con 0x7f00ec101f90 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.137+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec197a20 con 0x7f00ec101f90 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.137+0000 7f00d3fff700 1 -- 172.21.15.129:0/3589368957 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00dc003920 con 0x7f00ec101f90 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.137+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec197ee0 con 0x7f00ec101f90 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.138+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec103470 con 0x7f00ec101f90 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.138+0000 7f00d3fff700 1 -- 172.21.15.129:0/3589368957 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f00dc029070 con 0x7f00ec101f90 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.138+0000 7f00d3fff700 1 --2- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00d4039650 0x7f00d403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.138+0000 7f00d3fff700 1 -- 172.21.15.129:0/3589368957 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f00dc01e070 con 0x7f00ec101f90 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.138+0000 7f00ea59c700 1 --2- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00d4039650 0x7f00d403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.138+0000 7f00ea59c700 1 --2- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00d4039650 0x7f00d403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f00e0006fd0 tx=0x7f00e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:14.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.143+0000 7f00d3fff700 1 -- 172.21.15.129:0/3589368957 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f00dc013070 con 0x7f00ec101f90 2024-04-26T21:09:14.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:14.346+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) -- 0x7f00ec06b6d0 con 0x7f00d4039650 2024-04-26T21:09:16.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:16 smithi129 ceph-mon[29237]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:16.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:16 smithi129 ceph-mon[29237]: Deploying cephadm binary to smithi150 2024-04-26T21:09:16.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:16 smithi129 ceph-mon[29237]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:18 smithi129 ceph-mon[29237]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.557+0000 7f00d3fff700 1 -- 172.21.15.129:0/3589368957 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f00ec06b6d0 con 0x7f00d4039650 2024-04-26T21:09:18.558 INFO:teuthology.orchestra.run.smithi129.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-04-26T21:09:18.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00d4039650 msgr2=0x7f00d403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 --2- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00d4039650 0x7f00d403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f00e0006fd0 tx=0x7f00e0006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:18.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 msgr2=0x7f00ec1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 --2- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 0x7f00ec1974e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0056d0 tx=0x7f00dc0057b0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 shutdown_connections 2024-04-26T21:09:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 --2- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00d4039650 0x7f00d403bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 --2- 172.21.15.129:0/3589368957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00ec101f90 0x7f00ec1974e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 >> 172.21.15.129:0/3589368957 conn(0x7f00ec0fd860 msgr2=0x7f00ec103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 shutdown_connections 2024-04-26T21:09:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:18.559+0000 7f00f1123700 1 -- 172.21.15.129:0/3589368957 wait complete. 2024-04-26T21:09:19.191 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:09:19.422 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:19.814 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:19 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:19.814 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:19 smithi129 ceph-mon[29237]: Added host smithi150 2024-04-26T21:09:19.814 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:19 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:19.814 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:19 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:19.814 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:19 smithi129 ceph-mon[29237]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:20.718 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:20 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:20.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.928+0000 7f9408c49700 1 -- 172.21.15.129:0/3641400443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 msgr2=0x7f94041043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.928+0000 7f9408c49700 1 --2- 172.21.15.129:0/3641400443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 0x7f94041043b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009910 tx=0x7f93f4009c20 comp rx=0 tx=0).stop 2024-04-26T21:09:20.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.929+0000 7f9408c49700 1 -- 172.21.15.129:0/3641400443 shutdown_connections 2024-04-26T21:09:20.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.929+0000 7f9408c49700 1 --2- 172.21.15.129:0/3641400443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 0x7f94041043b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.929+0000 7f9408c49700 1 -- 172.21.15.129:0/3641400443 >> 172.21.15.129:0/3641400443 conn(0x7f94040fd820 msgr2=0x7f94040ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:20.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.929+0000 7f9408c49700 1 -- 172.21.15.129:0/3641400443 shutdown_connections 2024-04-26T21:09:20.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.929+0000 7f9408c49700 1 -- 172.21.15.129:0/3641400443 wait complete. 2024-04-26T21:09:20.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.929+0000 7f9408c49700 1 Processor -- start 2024-04-26T21:09:20.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.930+0000 7f9408c49700 1 -- start start 2024-04-26T21:09:20.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.930+0000 7f9408c49700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 0x7f940419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.930+0000 7f9408c49700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f940410d350 con 0x7f9404103fd0 2024-04-26T21:09:20.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.930+0000 7f940259c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 0x7f940419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.930+0000 7f940259c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 0x7f940419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36152/0 (socket says 172.21.15.129:36152) 2024-04-26T21:09:20.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.930+0000 7f940259c700 1 -- 172.21.15.129:0/1899377508 learned_addr learned my addr 172.21.15.129:0/1899377508 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:20.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.930+0000 7f940259c700 1 -- 172.21.15.129:0/1899377508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940419ebc0 con 0x7f9404103fd0 2024-04-26T21:09:20.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.930+0000 7f940259c700 1 --2- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 0x7f940419e680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93f40056d0 tx=0x7f93f40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:20.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.931+0000 7f93fbfff700 1 -- 172.21.15.129:0/1899377508 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f4026070 con 0x7f9404103fd0 2024-04-26T21:09:20.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.931+0000 7f93fbfff700 1 -- 172.21.15.129:0/1899377508 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f4004370 con 0x7f9404103fd0 2024-04-26T21:09:20.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.931+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940419ee50 con 0x7f9404103fd0 2024-04-26T21:09:20.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.931+0000 7f93fbfff700 1 -- 172.21.15.129:0/1899377508 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f40039b0 con 0x7f9404103fd0 2024-04-26T21:09:20.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.931+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9404198750 con 0x7f9404103fd0 2024-04-26T21:09:20.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.932+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f940406bb60 con 0x7f9404103fd0 2024-04-26T21:09:20.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.932+0000 7f93fbfff700 1 -- 172.21.15.129:0/1899377508 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f93f4029070 con 0x7f9404103fd0 2024-04-26T21:09:20.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.932+0000 7f93fbfff700 1 --2- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93f00396b0 0x7f93f003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.932+0000 7f93fbfff700 1 -- 172.21.15.129:0/1899377508 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f93f401e070 con 0x7f9404103fd0 2024-04-26T21:09:20.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.932+0000 7f93f9d9b700 1 --2- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93f00396b0 0x7f93f003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.933+0000 7f93f9d9b700 1 --2- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93f00396b0 0x7f93f003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006fd0 tx=0x7f93ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:20.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:20.937+0000 7f93fbfff700 1 -- 172.21.15.129:0/1899377508 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f93f4013070 con 0x7f9404103fd0 2024-04-26T21:09:21.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.141+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9404199c80 con 0x7f93f00396b0 2024-04-26T21:09:21.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.142+0000 7f93fbfff700 1 -- 172.21.15.129:0/1899377508 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f9404199c80 con 0x7f93f00396b0 2024-04-26T21:09:21.143 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:21.143 INFO:teuthology.orchestra.run.smithi129.stdout:[{"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-04-26T21:09:21.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93f00396b0 msgr2=0x7f93f003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 --2- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93f00396b0 0x7f93f003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006fd0 tx=0x7f93ec006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 msgr2=0x7f940419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 --2- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 0x7f940419e680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93f40056d0 tx=0x7f93f40057b0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 shutdown_connections 2024-04-26T21:09:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 --2- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93f00396b0 0x7f93f003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 --2- 172.21.15.129:0/1899377508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404103fd0 0x7f940419e680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 >> 172.21.15.129:0/1899377508 conn(0x7f94040fd820 msgr2=0x7f9404104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 shutdown_connections 2024-04-26T21:09:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:21.144+0000 7f9408c49700 1 -- 172.21.15.129:0/1899377508 wait complete. 2024-04-26T21:09:21.784 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-04-26T21:09:21.784 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:09:21.784 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:09:21.803 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:09:21.803 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:09:21.861 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-04-26T21:09:21.861 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch host add smithi171 2024-04-26T21:09:22.092 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:22 smithi129 ceph-mon[29237]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:09:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:22 smithi129 ceph-mon[29237]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:23.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.655+0000 7ff511bfa700 1 -- 172.21.15.129:0/2394720642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 msgr2=0x7ff50c107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.655+0000 7ff511bfa700 1 --2- 172.21.15.129:0/2394720642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 0x7ff50c107920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009910 tx=0x7ff4fc009c20 comp rx=0 tx=0).stop 2024-04-26T21:09:23.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.655+0000 7ff511bfa700 1 -- 172.21.15.129:0/2394720642 shutdown_connections 2024-04-26T21:09:23.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.655+0000 7ff511bfa700 1 --2- 172.21.15.129:0/2394720642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 0x7ff50c107920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.655+0000 7ff511bfa700 1 -- 172.21.15.129:0/2394720642 >> 172.21.15.129:0/2394720642 conn(0x7ff50c078410 msgr2=0x7ff50c078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:23.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.655+0000 7ff511bfa700 1 -- 172.21.15.129:0/2394720642 shutdown_connections 2024-04-26T21:09:23.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.655+0000 7ff511bfa700 1 -- 172.21.15.129:0/2394720642 wait complete. 2024-04-26T21:09:23.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.656+0000 7ff511bfa700 1 Processor -- start 2024-04-26T21:09:23.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.656+0000 7ff511bfa700 1 -- start start 2024-04-26T21:09:23.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.656+0000 7ff511bfa700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 0x7ff50c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.656+0000 7ff511bfa700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10b9b0 con 0x7ff50c107540 2024-04-26T21:09:23.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.656+0000 7ff50b7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 0x7ff50c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.656+0000 7ff50b7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 0x7ff50c1a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48752/0 (socket says 172.21.15.129:48752) 2024-04-26T21:09:23.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.656+0000 7ff50b7fe700 1 -- 172.21.15.129:0/1619838702 learned_addr learned my addr 172.21.15.129:0/1619838702 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:23.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.656+0000 7ff50b7fe700 1 -- 172.21.15.129:0/1619838702 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c1a0c10 con 0x7ff50c107540 2024-04-26T21:09:23.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.657+0000 7ff50b7fe700 1 --2- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 0x7ff50c1a06d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0056d0 tx=0x7ff4fc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.657+0000 7ff5097fa700 1 -- 172.21.15.129:0/1619838702 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4fc026070 con 0x7ff50c107540 2024-04-26T21:09:23.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.657+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c1a0ea0 con 0x7ff50c107540 2024-04-26T21:09:23.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.657+0000 7ff5097fa700 1 -- 172.21.15.129:0/1619838702 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc00bc00 con 0x7ff50c107540 2024-04-26T21:09:23.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.657+0000 7ff5097fa700 1 -- 172.21.15.129:0/1619838702 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4fc003920 con 0x7ff50c107540 2024-04-26T21:09:23.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.657+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c19a850 con 0x7ff50c107540 2024-04-26T21:09:23.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.658+0000 7ff5097fa700 1 -- 172.21.15.129:0/1619838702 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7ff4fc029070 con 0x7ff50c107540 2024-04-26T21:09:23.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.658+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50c102bb0 con 0x7ff50c107540 2024-04-26T21:09:23.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.658+0000 7ff5097fa700 1 --2- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff4f8039650 0x7ff4f803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.658+0000 7ff5097fa700 1 -- 172.21.15.129:0/1619838702 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff4fc01e070 con 0x7ff50c107540 2024-04-26T21:09:23.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.658+0000 7ff502ffd700 1 --2- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff4f8039650 0x7ff4f803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.658+0000 7ff502ffd700 1 --2- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff4f8039650 0x7ff4f803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4006fd0 tx=0x7ff4f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.663+0000 7ff5097fa700 1 -- 172.21.15.129:0/1619838702 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4fc013070 con 0x7ff50c107540 2024-04-26T21:09:23.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:23.866+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}) -- 0x7ff50c19bd80 con 0x7ff4f8039650 2024-04-26T21:09:24.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:24 smithi129 ceph-mon[29237]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:25 smithi129 ceph-mon[29237]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:09:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:26 smithi129 ceph-mon[29237]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-26T21:09:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:26 smithi129 ceph-mon[29237]: Deploying cephadm binary to smithi171 2024-04-26T21:09:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:26 smithi129 ceph-mon[29237]: Updating smithi150:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:26 smithi129 ceph-mon[29237]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:09:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:26 smithi129 ceph-mon[29237]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:27.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.929+0000 7ff5097fa700 1 -- 172.21.15.129:0/1619838702 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff50c19bd80 con 0x7ff4f8039650 2024-04-26T21:09:27.930 INFO:teuthology.orchestra.run.smithi129.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-04-26T21:09:27.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff4f8039650 msgr2=0x7ff4f803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 --2- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff4f8039650 0x7ff4f803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4006fd0 tx=0x7ff4f4006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:27.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 msgr2=0x7ff50c1a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 --2- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 0x7ff50c1a06d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0056d0 tx=0x7ff4fc0057b0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 shutdown_connections 2024-04-26T21:09:27.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 --2- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff4f8039650 0x7ff4f803bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 --2- 172.21.15.129:0/1619838702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff50c107540 0x7ff50c1a06d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 >> 172.21.15.129:0/1619838702 conn(0x7ff50c078410 msgr2=0x7ff50c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:27.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 shutdown_connections 2024-04-26T21:09:27.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:27.931+0000 7ff511bfa700 1 -- 172.21.15.129:0/1619838702 wait complete. 2024-04-26T21:09:28.502 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:28 smithi129 ceph-mon[29237]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:28.502 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:28 smithi129 ceph-mon[29237]: Updating smithi150:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:09:28.502 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:28.502 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:28.502 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:28.502 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:28.502 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:28.531 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:09:28.763 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:29.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:29 smithi129 ceph-mon[29237]: Added host smithi171 2024-04-26T21:09:29.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:29 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:29.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:29 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:30.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.328+0000 7f6fff58f700 1 -- 172.21.15.129:0/3206263692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 msgr2=0x7f6ff8108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.328+0000 7f6fff58f700 1 --2- 172.21.15.129:0/3206263692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 0x7f6ff8108190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009910 tx=0x7f6ff4009c20 comp rx=0 tx=0).stop 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.328+0000 7f6fff58f700 1 -- 172.21.15.129:0/3206263692 shutdown_connections 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.328+0000 7f6fff58f700 1 --2- 172.21.15.129:0/3206263692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 0x7f6ff8108190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.328+0000 7f6fff58f700 1 -- 172.21.15.129:0/3206263692 >> 172.21.15.129:0/3206263692 conn(0x7f6ff80fcfa0 msgr2=0x7f6ff80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.328+0000 7f6fff58f700 1 -- 172.21.15.129:0/3206263692 shutdown_connections 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.328+0000 7f6fff58f700 1 -- 172.21.15.129:0/3206263692 wait complete. 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.328+0000 7f6fff58f700 1 Processor -- start 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.328+0000 7f6fff58f700 1 -- start start 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6fff58f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 0x7f6ff819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6fff58f700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8113960 con 0x7f6ff8107db0 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6ffd32b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 0x7f6ff819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6ffd32b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 0x7f6ff819ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48776/0 (socket says 172.21.15.129:48776) 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6ffd32b700 1 -- 172.21.15.129:0/2031973991 learned_addr learned my addr 172.21.15.129:0/2031973991 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6ffd32b700 1 -- 172.21.15.129:0/2031973991 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff819f3a0 con 0x7f6ff8107db0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6ffd32b700 1 --2- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 0x7f6ff819ee60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4006010 tx=0x7f6ff4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6fee7fc700 1 -- 172.21.15.129:0/2031973991 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ff4026070 con 0x7f6ff8107db0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6fee7fc700 1 -- 172.21.15.129:0/2031973991 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff400bc00 con 0x7f6ff8107db0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff819f630 con 0x7f6ff8107db0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6fee7fc700 1 -- 172.21.15.129:0/2031973991 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ff4003910 con 0x7f6ff8107db0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.329+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff81a3730 con 0x7f6ff8107db0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.330+0000 7f6fee7fc700 1 -- 172.21.15.129:0/2031973991 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f6ff4029070 con 0x7f6ff8107db0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.330+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff81024a0 con 0x7f6ff8107db0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.330+0000 7f6fee7fc700 1 --2- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6fe4039650 0x7f6fe403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.330+0000 7f6fee7fc700 1 -- 172.21.15.129:0/2031973991 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6ff401e070 con 0x7f6ff8107db0 2024-04-26T21:09:30.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.330+0000 7f6ffcb2a700 1 --2- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6fe4039650 0x7f6fe403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.331+0000 7f6ffcb2a700 1 --2- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6fe4039650 0x7f6fe403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8006fd0 tx=0x7f6fe8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.335+0000 7f6fee7fc700 1 -- 172.21.15.129:0/2031973991 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6ff4013070 con 0x7f6ff8107db0 2024-04-26T21:09:30.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.539+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ff81a01c0 con 0x7f6fe4039650 2024-04-26T21:09:30.541 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:30 smithi129 ceph-mon[29237]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:30.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.541+0000 7f6fee7fc700 1 -- 172.21.15.129:0/2031973991 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f6ff81a01c0 con 0x7f6fe4039650 2024-04-26T21:09:30.542 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:09:30.542 INFO:teuthology.orchestra.run.smithi129.stdout:[{"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-04-26T21:09:30.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6fe4039650 msgr2=0x7f6fe403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 --2- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6fe4039650 0x7f6fe403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8006fd0 tx=0x7f6fe8006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:30.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 msgr2=0x7f6ff819ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 --2- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 0x7f6ff819ee60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4006010 tx=0x7f6ff4005780 comp rx=0 tx=0).stop 2024-04-26T21:09:30.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 shutdown_connections 2024-04-26T21:09:30.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 --2- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6fe4039650 0x7f6fe403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 --2- 172.21.15.129:0/2031973991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ff8107db0 0x7f6ff819ee60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 >> 172.21.15.129:0/2031973991 conn(0x7f6ff80fcfa0 msgr2=0x7f6ff80fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:30.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 shutdown_connections 2024-04-26T21:09:30.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:30.543+0000 7f6fff58f700 1 -- 172.21.15.129:0/2031973991 wait complete. 2024-04-26T21:09:31.208 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T21:09:31.208 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T21:09:31.444 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:31.775 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:31 smithi129 ceph-mon[29237]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:09:31.775 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:31 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:31.776 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:31 smithi129 ceph-mon[29237]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 -- 172.21.15.129:0/2292152503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 msgr2=0x7f6698102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 --2- 172.21.15.129:0/2292152503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 0x7f6698102390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6688009910 tx=0x7f6688009c20 comp rx=0 tx=0).stop 2024-04-26T21:09:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 -- 172.21.15.129:0/2292152503 shutdown_connections 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 --2- 172.21.15.129:0/2292152503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 0x7f6698102390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 -- 172.21.15.129:0/2292152503 >> 172.21.15.129:0/2292152503 conn(0x7f66980fd840 msgr2=0x7f66980ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 -- 172.21.15.129:0/2292152503 shutdown_connections 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 -- 172.21.15.129:0/2292152503 wait complete. 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 Processor -- start 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 -- start start 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 0x7f669819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.977+0000 7f66a05da700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f669810c230 con 0x7f6698101fb0 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f669e376700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 0x7f669819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f669e376700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 0x7f669819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52388/0 (socket says 172.21.15.129:52388) 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f669e376700 1 -- 172.21.15.129:0/510689493 learned_addr learned my addr 172.21.15.129:0/510689493 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f669e376700 1 -- 172.21.15.129:0/510689493 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669819eaf0 con 0x7f6698101fb0 2024-04-26T21:09:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f669e376700 1 --2- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 0x7f669819e5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66880056d0 tx=0x7f66880057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:32.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f668f7fe700 1 -- 172.21.15.129:0/510689493 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6688026070 con 0x7f6698101fb0 2024-04-26T21:09:32.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f668f7fe700 1 -- 172.21.15.129:0/510689493 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668800bba0 con 0x7f6698101fb0 2024-04-26T21:09:32.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669819ede0 con 0x7f6698101fb0 2024-04-26T21:09:32.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f668f7fe700 1 -- 172.21.15.129:0/510689493 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6688003920 con 0x7f6698101fb0 2024-04-26T21:09:32.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.978+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6698198850 con 0x7f6698101fb0 2024-04-26T21:09:32.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.979+0000 7f668f7fe700 1 -- 172.21.15.129:0/510689493 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f6688029070 con 0x7f6698101fb0 2024-04-26T21:09:32.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.979+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6698103490 con 0x7f6698101fb0 2024-04-26T21:09:32.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.979+0000 7f668f7fe700 1 --2- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6684039650 0x7f668403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.979+0000 7f668f7fe700 1 -- 172.21.15.129:0/510689493 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f668801e070 con 0x7f6698101fb0 2024-04-26T21:09:32.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.979+0000 7f669db75700 1 --2- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6684039650 0x7f668403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.980+0000 7f669db75700 1 --2- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6684039650 0x7f668403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6694006fd0 tx=0x7f6694006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:32.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:32.985+0000 7f668f7fe700 1 -- 172.21.15.129:0/510689493 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6688013070 con 0x7f6698101fb0 2024-04-26T21:09:33.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.197+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f66980008d0 con 0x7f6698101fb0 2024-04-26T21:09:33.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.251+0000 7f668f7fe700 1 -- 172.21.15.129:0/510689493 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f6688030090 con 0x7f6698101fb0 2024-04-26T21:09:33.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6684039650 msgr2=0x7f668403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 --2- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6684039650 0x7f668403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6694006fd0 tx=0x7f6694006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:33.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 msgr2=0x7f669819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 --2- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 0x7f669819e5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66880056d0 tx=0x7f66880057b0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 shutdown_connections 2024-04-26T21:09:33.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 --2- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6684039650 0x7f668403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 --2- 172.21.15.129:0/510689493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6698101fb0 0x7f669819e5b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 >> 172.21.15.129:0/510689493 conn(0x7f66980fd840 msgr2=0x7f669806b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 shutdown_connections 2024-04-26T21:09:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:33.255+0000 7f66a05da700 1 -- 172.21.15.129:0/510689493 wait complete. 2024-04-26T21:09:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:adjusted tunables profile to default 2024-04-26T21:09:33.469 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:33 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/510689493' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:09:33.951 INFO:tasks.cephadm:Adding mon.a on smithi129 2024-04-26T21:09:33.951 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-04-26T21:09:33.951 INFO:tasks.cephadm:Adding mon.c on smithi171 2024-04-26T21:09:33.951 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi129:172.21.15.129=a;smithi150:172.21.15.150=b;smithi171:172.21.15.171=c' 2024-04-26T21:09:34.184 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:09:34.233 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:09:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/510689493' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:09:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:09:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:34.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:35.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:35 smithi129 ceph-mon[29237]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-26T21:09:35.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:35 smithi129 ceph-mon[29237]: Updating smithi171:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:35.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:35.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:35.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:35.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.832+0000 7f8f98dfc700 1 -- 172.21.15.171:0/808358296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 msgr2=0x7f8f94102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.832+0000 7f8f98dfc700 1 --2- 172.21.15.171:0/808358296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 0x7f8f94102370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00b0a0 tx=0x7f8f8c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.832+0000 7f8f98dfc700 1 -- 172.21.15.171:0/808358296 shutdown_connections 2024-04-26T21:09:35.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.832+0000 7f8f98dfc700 1 --2- 172.21.15.171:0/808358296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 0x7f8f94102370 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.832+0000 7f8f98dfc700 1 -- 172.21.15.171:0/808358296 >> 172.21.15.171:0/808358296 conn(0x7f8f940fd860 msgr2=0x7f8f940ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:35.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.832+0000 7f8f98dfc700 1 -- 172.21.15.171:0/808358296 shutdown_connections 2024-04-26T21:09:35.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.832+0000 7f8f98dfc700 1 -- 172.21.15.171:0/808358296 wait complete. 2024-04-26T21:09:35.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.833+0000 7f8f98dfc700 1 Processor -- start 2024-04-26T21:09:35.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.833+0000 7f8f98dfc700 1 -- start start 2024-04-26T21:09:35.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.833+0000 7f8f98dfc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 0x7f8f9419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.833+0000 7f8f98dfc700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9410c210 con 0x7f8f94101f90 2024-04-26T21:09:35.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f9259c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 0x7f8f9419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f9259c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 0x7f8f9419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.171:44946/0 (socket says 172.21.15.171:44946) 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f9259c700 1 -- 172.21.15.171:0/2452544058 learned_addr learned my addr 172.21.15.171:0/2452544058 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f9259c700 1 -- 172.21.15.171:0/2452544058 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9419e9a0 con 0x7f8f94101f90 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f9259c700 1 --2- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 0x7f8f9419e460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c006130 tx=0x7f8f8c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f837fe700 1 -- 172.21.15.171:0/2452544058 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f8c021020 con 0x7f8f94101f90 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9419ec30 con 0x7f8f94101f90 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f837fe700 1 -- 172.21.15.171:0/2452544058 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f8c0079b0 con 0x7f8f94101f90 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f837fe700 1 -- 172.21.15.171:0/2452544058 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f8c017040 con 0x7f8f94101f90 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.834+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f94198620 con 0x7f8f94101f90 2024-04-26T21:09:35.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.835+0000 7f8f837fe700 1 -- 172.21.15.171:0/2452544058 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f8f8c024020 con 0x7f8f94101f90 2024-04-26T21:09:35.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.835+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f94103470 con 0x7f8f94101f90 2024-04-26T21:09:35.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.835+0000 7f8f837fe700 1 --2- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8f7c03da60 0x7f8f7c03ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.835+0000 7f8f837fe700 1 -- 172.21.15.171:0/2452544058 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f8c00f050 con 0x7f8f94101f90 2024-04-26T21:09:35.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.836+0000 7f8f91d9b700 1 --2- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8f7c03da60 0x7f8f7c03ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.836+0000 7f8f91d9b700 1 --2- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8f7c03da60 0x7f8f7c03ff20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006fd0 tx=0x7f8f88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:35.841 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:35.841+0000 7f8f837fe700 1 -- 172.21.15.171:0/2452544058 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8f8c02b020 con 0x7f8f94101f90 2024-04-26T21:09:36.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.053+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi129:172.21.15.129=a;smithi150:172.21.15.150=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}) -- 0x7f8f94199b50 con 0x7f8f7c03da60 2024-04-26T21:09:36.060 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.060+0000 7f8f837fe700 1 -- 172.21.15.171:0/2452544058 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8f94199b50 con 0x7f8f7c03da60 2024-04-26T21:09:36.061 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mon update... 2024-04-26T21:09:36.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8f7c03da60 msgr2=0x7f8f7c03ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 --2- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8f7c03da60 0x7f8f7c03ff20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006fd0 tx=0x7f8f88006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:36.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 msgr2=0x7f8f9419e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 --2- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 0x7f8f9419e460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c006130 tx=0x7f8f8c0097d0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 shutdown_connections 2024-04-26T21:09:36.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 --2- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8f7c03da60 0x7f8f7c03ff20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 --2- 172.21.15.171:0/2452544058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f94101f90 0x7f8f9419e460 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 >> 172.21.15.171:0/2452544058 conn(0x7f8f940fd860 msgr2=0x7f8f9406b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:36.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 shutdown_connections 2024-04-26T21:09:36.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:36.062+0000 7f8f98dfc700 1 -- 172.21.15.171:0/2452544058 wait complete. 2024-04-26T21:09:36.662 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T21:09:36.665 DEBUG:teuthology.orchestra.run.smithi171:mon.c> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T21:09:36.667 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:09:36.667 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:09:36.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:09:36.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: Updating smithi171:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:09:36.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:36.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:36.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:36.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:36.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:36.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:36.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:09:36.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:37.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:37 smithi129 ceph-mon[29237]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi129:172.21.15.129=a;smithi150:172.21.15.150=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:37.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:37 smithi129 ceph-mon[29237]: Saving service mon spec with placement smithi129:172.21.15.129=a;smithi150:172.21.15.150=b;smithi171:172.21.15.171=c;count:3 2024-04-26T21:09:37.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:37 smithi129 ceph-mon[29237]: Deploying daemon mon.c on smithi171 2024-04-26T21:09:37.729 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:38 smithi129 ceph-mon[29237]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:39.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:39 smithi171 systemd[1]: Starting Ceph mon.c for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.383+0000 7f62f06e1700 1 -- 172.21.15.171:0/3950632903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 msgr2=0x7f62e81043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.383+0000 7f62f06e1700 1 --2- 172.21.15.171:0/3950632903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 0x7f62e81043e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009910 tx=0x7f62e4009c20 comp rx=0 tx=0).stop 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.383+0000 7f62f06e1700 1 -- 172.21.15.171:0/3950632903 shutdown_connections 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.383+0000 7f62f06e1700 1 --2- 172.21.15.171:0/3950632903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 0x7f62e81043e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.383+0000 7f62f06e1700 1 -- 172.21.15.171:0/3950632903 >> 172.21.15.171:0/3950632903 conn(0x7f62e80789f0 msgr2=0x7f62e80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.383+0000 7f62f06e1700 1 -- 172.21.15.171:0/3950632903 shutdown_connections 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.383+0000 7f62f06e1700 1 -- 172.21.15.171:0/3950632903 wait complete. 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.384+0000 7f62f06e1700 1 Processor -- start 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.384+0000 7f62f06e1700 1 -- start start 2024-04-26T21:09:40.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.384+0000 7f62f06e1700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 0x7f62e819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.384+0000 7f62f06e1700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62e810d350 con 0x7f62e8104000 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.384+0000 7f62ee47d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 0x7f62e819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.384+0000 7f62ee47d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 0x7f62e819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.171:56204/0 (socket says 172.21.15.171:56204) 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.384+0000 7f62ee47d700 1 -- 172.21.15.171:0/1123548765 learned_addr learned my addr 172.21.15.171:0/1123548765 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.385+0000 7f62ee47d700 1 -- 172.21.15.171:0/1123548765 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e819eb50 con 0x7f62e8104000 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.385+0000 7f62ee47d700 1 --2- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 0x7f62e819e610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62e40056d0 tx=0x7f62e40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.385+0000 7f62df7fe700 1 -- 172.21.15.171:0/1123548765 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e4026070 con 0x7f62e8104000 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.385+0000 7f62df7fe700 1 -- 172.21.15.171:0/1123548765 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e4004370 con 0x7f62e8104000 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.385+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e819ede0 con 0x7f62e8104000 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.385+0000 7f62df7fe700 1 -- 172.21.15.171:0/1123548765 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e40039b0 con 0x7f62e8104000 2024-04-26T21:09:40.387 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.385+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e81987e0 con 0x7f62e8104000 2024-04-26T21:09:40.388 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.386+0000 7f62df7fe700 1 -- 172.21.15.171:0/1123548765 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f62e4029070 con 0x7f62e8104000 2024-04-26T21:09:40.388 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.386+0000 7f62df7fe700 1 --2- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62d4039650 0x7f62d403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.388 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.386+0000 7f62df7fe700 1 -- 172.21.15.171:0/1123548765 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f62e401e070 con 0x7f62e8104000 2024-04-26T21:09:40.388 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.386+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e806b0f0 con 0x7f62e8104000 2024-04-26T21:09:40.388 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.386+0000 7f62edc7c700 1 --2- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62d4039650 0x7f62d403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.388 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.387+0000 7f62edc7c700 1 --2- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62d4039650 0x7f62d403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f62d8006fd0 tx=0x7f62d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:40.392 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.392+0000 7f62df7fe700 1 -- 172.21.15.171:0/1123548765 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f62e4030020 con 0x7f62e8104000 2024-04-26T21:09:40.691 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.689+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f62e8064a80 con 0x7f62e8104000 2024-04-26T21:09:40.691 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.690+0000 7f62df7fe700 1 -- 172.21.15.171:0/1123548765 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f62e4013070 con 0x7f62e8104000 2024-04-26T21:09:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:09:40.691 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":1,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","modified":"2024-04-26T21:07:47.959085Z","created":"2024-04-26T21:07:47.959085Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62d4039650 msgr2=0x7f62d403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 --2- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62d4039650 0x7f62d403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f62d8006fd0 tx=0x7f62d8006e40 comp rx=0 tx=0).stop 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 msgr2=0x7f62e819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 --2- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 0x7f62e819e610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62e40056d0 tx=0x7f62e40057b0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 shutdown_connections 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 --2- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62d4039650 0x7f62d403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 --2- 172.21.15.171:0/1123548765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e8104000 0x7f62e819e610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 >> 172.21.15.171:0/1123548765 conn(0x7f62e80789f0 msgr2=0x7f62e8104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 shutdown_connections 2024-04-26T21:09:40.694 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:40.693+0000 7f62f06e1700 1 -- 172.21.15.171:0/1123548765 wait complete. 2024-04-26T21:09:40.695 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 1 2024-04-26T21:09:40.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:40 smithi129 ceph-mon[29237]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:40.821 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 podman[33444]: 2024-04-26 21:09:40.452163264 +0000 UTC m=+0.017918100 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:09:40.822 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 podman[33444]: 2024-04-26 21:09:40.557323477 +0000 UTC m=+0.123078288 container create e0a14f2dafd7532d960af766102b4974beb1ec4870fc6c857903910c27cbad00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:09:40.822 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 podman[33444]: 2024-04-26 21:09:40.793924075 +0000 UTC m=+0.359678893 container init e0a14f2dafd7532d960af766102b4974beb1ec4870fc6c857903910c27cbad00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:09:40.822 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 podman[33444]: 2024-04-26 21:09:40.797458106 +0000 UTC m=+0.363212915 container start e0a14f2dafd7532d960af766102b4974beb1ec4870fc6c857903910c27cbad00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:09:41.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:09:41.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:09:41.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pidfile_write: ignore empty --pid-file 2024-04-26T21:09:41.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: load: jerasure load: lrc 2024-04-26T21:09:41.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:09:41.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Git sha 0 2024-04-26T21:09:41.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: DB SUMMARY 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: DB Session ID: TXV7D3ELGTHYCMFG2I6F 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:09:41.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.env: 0x559339353cc0 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.info_log: 0x55933b00fd80 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.statistics: (nil) 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.use_fsync: 0 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:09:41.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.db_log_dir: 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.wal_dir: 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:09:41.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.write_buffer_manager: 0x55933afe10e0 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.unordered_write: 0 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.row_cache: None 2024-04-26T21:09:41.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.wal_filter: None 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.wal_compression: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:09:41.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:09:41.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:09:41.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:09:41.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:09:41.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:09:41.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:09:41.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:09:41.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:09:41.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:09:41.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_open_files: -1 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Compression algorithms supported: 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:09:41.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: kZSTD supported: 0 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.merge_operator: 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_filter: None 2024-04-26T21:09:41.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55933b00ff40) 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_type: 0 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_index_type: 0 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_shortening: 1 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: checksum: 4 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: no_block_cache: 0 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache: 0x55933b0131f0 2024-04-26T21:09:41.243 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_options: 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: capacity : 536870912 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_shard_bits : 4 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: strict_capacity_limit : 0 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_compressed: (nil) 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: persistent_cache: (nil) 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size: 4096 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size_deviation: 10 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_restart_interval: 16 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_block_restart_interval: 1 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: metadata_block_size: 4096 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: partition_filters: 0 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: use_delta_encoding: 1 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: filter_policy: bloomfilter 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: whole_key_filtering: 1 2024-04-26T21:09:41.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: verify_compression: 0 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: format_version: 5 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_index_compression: 1 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_align: 0 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: max_auto_readahead_size: 262144 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: prepopulate_block_cache: 0 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression: NoCompression 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:09:41.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.num_levels: 7 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:09:41.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:09:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:09:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.ttl: 2592000 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:09:41.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bbdb3b73-c991-483e-b639-8d61a60340ab 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165780827380, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165780827806, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714165780, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bbdb3b73-c991-483e-b639-8d61a60340ab", "db_session_id": "TXV7D3ELGTHYCMFG2I6F", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:09:41.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165780827884, "job": 1, "event": "recovery_finished"} 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55933b130000 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: DB pointer 0x55933b11c000 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** DB Stats ** 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:09:41.252 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-26T21:09:41.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache BinnedLRUCache@0x55933b0131f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-26T21:09:41.254 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: using public_addr v2:172.21.15.171:0/0 -> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: starting mon.c rank -1 at public addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] at bind addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(???) e0 preinit fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: e1 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: legacy client fscid: -1 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: No filesystems configured 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T21:09:41.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:41.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Deploying cephadm binary to smithi171 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Updating smithi150:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Updating smithi150:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Added host smithi171 2024-04-26T21:09:41.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/510689493' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/510689493' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Updating smithi171:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Updating smithi171:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:41.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi129:172.21.15.129=a;smithi150:172.21.15.150=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Saving service mon spec with placement smithi129:172.21.15.129=a;smithi150:172.21.15.150=b;smithi171:172.21.15.171=c;count:3 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: Deploying daemon mon.c on smithi171 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 ceph-mon[33478]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 bash[33444]: e0a14f2dafd7532d960af766102b4974beb1ec4870fc6c857903910c27cbad00 2024-04-26T21:09:41.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:40 smithi171 systemd[1]: Started Ceph mon.c for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:09:42.468 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:09:42.468 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:09:42.712 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:09:46.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:09:46.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: mon.a calling monitor election 2024-04-26T21:09:46.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:46.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: mon.c calling monitor election 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: monmap epoch 2 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: last_changed 2024-04-26T21:09:40.852803+0000 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: min_mon_release 19 (squid) 2024-04-26T21:09:46.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: election_strategy: 1 2024-04-26T21:09:46.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:09:46.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-26T21:09:46.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: fsmap 2024-04-26T21:09:46.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:46.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: mgrmap e13: x(active, since 50s) 2024-04-26T21:09:46.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: overall HEALTH_OK 2024-04-26T21:09:46.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:46.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: mon.a calling monitor election 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: mon.c calling monitor election 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: monmap epoch 2 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: last_changed 2024-04-26T21:09:40.852803+0000 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: min_mon_release 19 (squid) 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: election_strategy: 1 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: fsmap 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: mgrmap e13: x(active, since 50s) 2024-04-26T21:09:46.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: overall HEALTH_OK 2024-04-26T21:09:46.244 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:46.244 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:47.124 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.122+0000 7f898bac1700 1 -- 172.21.15.171:0/3646738222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89700035c0 msgr2=0x7f8970005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.124 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.122+0000 7f898bac1700 1 --2- 172.21.15.171:0/3646738222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89700035c0 0x7f8970005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8984102a50 tx=0x7f898000de80 comp rx=0 tx=0).stop 2024-04-26T21:09:47.124 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.123+0000 7f898bac1700 1 -- 172.21.15.171:0/3646738222 shutdown_connections 2024-04-26T21:09:47.124 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.123+0000 7f898bac1700 1 --2- 172.21.15.171:0/3646738222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89700035c0 0x7f8970005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.124 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.123+0000 7f898bac1700 1 -- 172.21.15.171:0/3646738222 >> 172.21.15.171:0/3646738222 conn(0x7f89840784d0 msgr2=0x7f89840788e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:47.124 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.123+0000 7f898bac1700 1 -- 172.21.15.171:0/3646738222 shutdown_connections 2024-04-26T21:09:47.124 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.123+0000 7f898bac1700 1 -- 172.21.15.171:0/3646738222 wait complete. 2024-04-26T21:09:47.126 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898bac1700 1 Processor -- start 2024-04-26T21:09:47.126 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898bac1700 1 -- start start 2024-04-26T21:09:47.126 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898bac1700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89700035c0 0x7f89841a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.126 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898bac1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 0x7f898419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.126 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898bac1700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8984108ca0 con 0x7f89700035c0 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898bac1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8984108b20 con 0x7f89841a0ce0 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898905c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 0x7f898419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898985d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89700035c0 0x7f89841a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898905c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 0x7f898419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:33878/0 (socket says 172.21.15.171:33878) 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898905c700 1 -- 172.21.15.171:0/2381124080 learned_addr learned my addr 172.21.15.171:0/2381124080 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898905c700 1 -- 172.21.15.171:0/2381124080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 msgr2=0x7f898419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898905c700 1 -- 172.21.15.171:0/2381124080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 msgr2=0x7f898419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898905c700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 0x7f898419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898905c700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 0x7f898419a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898985d700 1 -- 172.21.15.171:0/2381124080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 msgr2=0x7f898419a9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898985d700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 0x7f898419a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.127 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.124+0000 7f898985d700 1 -- 172.21.15.171:0/2381124080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89841a1230 con 0x7f89700035c0 2024-04-26T21:09:47.128 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.125+0000 7f898985d700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89700035c0 0x7f89841a07a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8984102a50 tx=0x7f8980003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:47.128 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.125+0000 7f897affd700 1 -- 172.21.15.171:0/2381124080 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8980005470 con 0x7f89700035c0 2024-04-26T21:09:47.128 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.125+0000 7f897affd700 1 -- 172.21.15.171:0/2381124080 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8980005610 con 0x7f89700035c0 2024-04-26T21:09:47.128 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.125+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898419aef0 con 0x7f89700035c0 2024-04-26T21:09:47.128 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.125+0000 7f897affd700 1 -- 172.21.15.171:0/2381124080 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f898000e780 con 0x7f89700035c0 2024-04-26T21:09:47.128 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.125+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898419b500 con 0x7f89700035c0 2024-04-26T21:09:47.129 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.126+0000 7f897affd700 1 -- 172.21.15.171:0/2381124080 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f898000ea30 con 0x7f89700035c0 2024-04-26T21:09:47.129 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.126+0000 7f897affd700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8968039460 0x7f896803b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.129 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.126+0000 7f897affd700 1 -- 172.21.15.171:0/2381124080 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8980016070 con 0x7f89700035c0 2024-04-26T21:09:47.129 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.126+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898419b7b0 con 0x7f89700035c0 2024-04-26T21:09:47.129 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.126+0000 7f898905c700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8968039460 0x7f896803b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.129 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.127+0000 7f898905c700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8968039460 0x7f896803b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8974009180 tx=0x7f8974008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:47.133 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.132+0000 7f897affd700 1 -- 172.21.15.171:0/2381124080 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f898000edc0 con 0x7f89700035c0 2024-04-26T21:09:47.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:46 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:47.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:46 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:09:47.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:46 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:47.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:46 smithi129 ceph-mon[29237]: Deploying daemon mon.b on smithi150 2024-04-26T21:09:47.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:46 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:47.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:46 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:47.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:46 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:09:47.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:46 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:47.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:46 smithi171 ceph-mon[33478]: Deploying daemon mon.b on smithi150 2024-04-26T21:09:47.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:46 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:47.416 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.415+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8984064a80 con 0x7f89700035c0 2024-04-26T21:09:47.416 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.416+0000 7f897affd700 1 -- 172.21.15.171:0/2381124080 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f8980018c60 con 0x7f89700035c0 2024-04-26T21:09:47.417 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:09:47.417 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","modified":"2024-04-26T21:09:40.852803Z","created":"2024-04-26T21:07:47.959085Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:09:47.418 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8968039460 msgr2=0x7f896803b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8968039460 0x7f896803b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8974009180 tx=0x7f8974008040 comp rx=0 tx=0).stop 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89700035c0 msgr2=0x7f89841a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89700035c0 0x7f89841a07a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8984102a50 tx=0x7f8980003ce0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 shutdown_connections 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8968039460 0x7f896803b920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89700035c0 0x7f89841a07a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 --2- 172.21.15.171:0/2381124080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89841a0ce0 0x7f898419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 >> 172.21.15.171:0/2381124080 conn(0x7f89840784d0 msgr2=0x7f8984100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 shutdown_connections 2024-04-26T21:09:47.419 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:47.418+0000 7f898bac1700 1 -- 172.21.15.171:0/2381124080 wait complete. 2024-04-26T21:09:47.420 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-04-26T21:09:47.964 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:47 smithi171 ceph-mon[33478]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:47.964 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:47 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/2381124080' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:09:48.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:47 smithi129 ceph-mon[29237]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:48.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:47 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/2381124080' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:09:48.992 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:09:48.993 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:09:49.242 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:09:50.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:49 smithi150 systemd[1]: Started Ceph mon.b for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:09:55.132 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.130+0000 7fd4bffb5700 1 -- 172.21.15.171:0/863731763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a40035c0 msgr2=0x7fd4a4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.132 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.130+0000 7fd4bffb5700 1 --2- 172.21.15.171:0/863731763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a40035c0 0x7fd4a4005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd4b806ae80 tx=0x7fd4b400fe60 comp rx=0 tx=0).stop 2024-04-26T21:09:55.132 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.130+0000 7fd4bffb5700 1 -- 172.21.15.171:0/863731763 shutdown_connections 2024-04-26T21:09:55.132 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.130+0000 7fd4bffb5700 1 --2- 172.21.15.171:0/863731763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a40035c0 0x7fd4a4005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.132 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.130+0000 7fd4bffb5700 1 -- 172.21.15.171:0/863731763 >> 172.21.15.171:0/863731763 conn(0x7fd4b80fd900 msgr2=0x7fd4b80ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.132 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.130+0000 7fd4bffb5700 1 -- 172.21.15.171:0/863731763 shutdown_connections 2024-04-26T21:09:55.132 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.130+0000 7fd4bffb5700 1 -- 172.21.15.171:0/863731763 wait complete. 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: mon.a calling monitor election 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: mon.c calling monitor election 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: monmap epoch 3 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: last_changed 2024-04-26T21:09:49.775701+0000 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: min_mon_release 19 (squid) 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: election_strategy: 1 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: fsmap 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: mgrmap e13: x(active, since 59s) 2024-04-26T21:09:55.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: overall HEALTH_OK 2024-04-26T21:09:55.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:55.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:55.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:55.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:55.139 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.131+0000 7fd4bffb5700 1 Processor -- start 2024-04-26T21:09:55.139 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.131+0000 7fd4bffb5700 1 -- start start 2024-04-26T21:09:55.139 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.131+0000 7fd4bffb5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40035c0 0x7fd4b819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.139 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bffb5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 0x7fd4b81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.139 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bffb5700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4b8198f10 0x7fd4b81993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.139 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bffb5700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b810ae10 con 0x7fd4b8198f10 2024-04-26T21:09:55.139 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bffb5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b810ac90 con 0x7fd4a40035c0 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bffb5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b810af90 con 0x7fd4b819ed50 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bd550700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 0x7fd4b81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bd550700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 0x7fd4b81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:46170/0 (socket says 172.21.15.171:46170) 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bd550700 1 -- 172.21.15.171:0/1834440049 learned_addr learned my addr 172.21.15.171:0/1834440049 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4be552700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4b8198f10 0x7fd4b81993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bd550700 1 -- 172.21.15.171:0/1834440049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 msgr2=0x7fd4b81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bd550700 1 -- 172.21.15.171:0/1834440049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 msgr2=0x7fd4b81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bd550700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 0x7fd4b81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bd550700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 0x7fd4b81989d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bdd51700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40035c0 0x7fd4b819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bdd51700 1 -- 172.21.15.171:0/1834440049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40035c0 msgr2=0x7fd4b819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bdd51700 1 -- 172.21.15.171:0/1834440049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40035c0 msgr2=0x7fd4b819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:09:55.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bdd51700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40035c0 0x7fd4b819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4be552700 1 -- 172.21.15.171:0/1834440049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 msgr2=0x7fd4b81989d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4be552700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 0x7fd4b81989d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4be552700 1 -- 172.21.15.171:0/1834440049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40035c0 msgr2=0x7fd4b819e810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4bdd51700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40035c0 0x7fd4b819e810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4be552700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40035c0 0x7fd4b819e810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4be552700 1 -- 172.21.15.171:0/1834440049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b81a56e0 con 0x7fd4b8198f10 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.132+0000 7fd4be552700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4b8198f10 0x7fd4b81993c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000bd50 tx=0x7fd4b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.133+0000 7fd4aeffd700 1 -- 172.21.15.171:0/1834440049 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0019070 con 0x7fd4b8198f10 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.133+0000 7fd4aeffd700 1 -- 172.21.15.171:0/1834440049 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b000cd10 con 0x7fd4b8198f10 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.133+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b81a5910 con 0x7fd4b8198f10 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.133+0000 7fd4aeffd700 1 -- 172.21.15.171:0/1834440049 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0012700 con 0x7fd4b8198f10 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.133+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b81a5ea0 con 0x7fd4b8198f10 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.134+0000 7fd4aeffd700 1 -- 172.21.15.171:0/1834440049 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fd4b001c070 con 0x7fd4b8198f10 2024-04-26T21:09:55.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.134+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b8064a80 con 0x7fd4b8198f10 2024-04-26T21:09:55.142 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.134+0000 7fd4aeffd700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd49c039ae0 0x7fd49c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.142 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.134+0000 7fd4aeffd700 1 -- 172.21.15.171:0/1834440049 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd4b004d080 con 0x7fd4b8198f10 2024-04-26T21:09:55.142 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.134+0000 7fd4bdd51700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd49c039ae0 0x7fd49c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.142 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.135+0000 7fd4bdd51700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd49c039ae0 0x7fd49c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4005b40 tx=0x7fd4b4005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.142 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.139+0000 7fd4aeffd700 1 -- 172.21.15.171:0/1834440049 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd4b000e0a0 con 0x7fd4b8198f10 2024-04-26T21:09:55.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:09:55.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: mon.a calling monitor election 2024-04-26T21:09:55.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: mon.c calling monitor election 2024-04-26T21:09:55.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:55.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: monmap epoch 3 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: last_changed 2024-04-26T21:09:49.775701+0000 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: min_mon_release 19 (squid) 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: election_strategy: 1 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:09:55.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-26T21:09:55.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-26T21:09:55.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: fsmap 2024-04-26T21:09:55.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:55.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: mgrmap e13: x(active, since 59s) 2024-04-26T21:09:55.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: overall HEALTH_OK 2024-04-26T21:09:55.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:55.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:55.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:55.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:55.454 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.453+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd4b81a6190 con 0x7fd4b8198f10 2024-04-26T21:09:55.455 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.454+0000 7fd4aeffd700 1 -- 172.21.15.171:0/1834440049 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fd4b0007290 con 0x7fd4b8198f10 2024-04-26T21:09:55.455 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-26T21:09:55.455 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":3,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","modified":"2024-04-26T21:09:49.775701Z","created":"2024-04-26T21:07:47.959085Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:09:55.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd49c039ae0 msgr2=0x7fd49c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd49c039ae0 0x7fd49c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4005b40 tx=0x7fd4b4005a90 comp rx=0 tx=0).stop 2024-04-26T21:09:55.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4b8198f10 msgr2=0x7fd4b81993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4b8198f10 0x7fd4b81993c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000bd50 tx=0x7fd4b000be30 comp rx=0 tx=0).stop 2024-04-26T21:09:55.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 shutdown_connections 2024-04-26T21:09:55.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd49c039ae0 0x7fd49c03bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.458 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4b8198f10 0x7fd4b81993c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.458 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40035c0 0x7fd4b819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.458 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 --2- 172.21.15.171:0/1834440049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4b819ed50 0x7fd4b81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.458 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 >> 172.21.15.171:0/1834440049 conn(0x7fd4b80fd900 msgr2=0x7fd4b8104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.458 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 shutdown_connections 2024-04-26T21:09:55.458 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:09:55.456+0000 7fd4bffb5700 1 -- 172.21.15.171:0/1834440049 wait complete. 2024-04-26T21:09:55.458 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 3 2024-04-26T21:09:56.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:56.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:55 smithi171 ceph-mon[33478]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:56.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:56.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:55 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/1834440049' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:09:56.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:56.055 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T21:09:56.056 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T21:09:56.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:56.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:55 smithi129 ceph-mon[29237]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:56.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:56.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:55 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/1834440049' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:09:56.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:56.305 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:57.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:09:57.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:57.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: mon.a calling monitor election 2024-04-26T21:09:57.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: mon.c calling monitor election 2024-04-26T21:09:57.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: monmap epoch 3 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: last_changed 2024-04-26T21:09:49.775701+0000 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: min_mon_release 19 (squid) 2024-04-26T21:09:57.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: election_strategy: 1 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: fsmap 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: mgrmap e13: x(active, since 59s) 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: overall HEALTH_OK 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:57.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='client.? 172.21.15.171:0/1834440049' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:09:57.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: mon.b calling monitor election 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: mon.b calling monitor election 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: mon.c calling monitor election 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: mon.a calling monitor election 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: monmap epoch 3 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: last_changed 2024-04-26T21:09:49.775701+0000 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: min_mon_release 19 (squid) 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: election_strategy: 1 2024-04-26T21:09:57.546 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: fsmap 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: mgrmap e13: x(active, since 61s) 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: overall HEALTH_OK 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:57.547 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: mon.b calling monitor election 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: mon.b calling monitor election 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: mon.c calling monitor election 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: mon.a calling monitor election 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: monmap epoch 3 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: last_changed 2024-04-26T21:09:49.775701+0000 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:09:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: min_mon_release 19 (squid) 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: election_strategy: 1 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: fsmap 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: mgrmap e13: x(active, since 61s) 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: overall HEALTH_OK 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:57.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: mon.b calling monitor election 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: mon.b calling monitor election 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: mon.c calling monitor election 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: mon.a calling monitor election 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: monmap epoch 3 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: last_changed 2024-04-26T21:09:49.775701+0000 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: created 2024-04-26T21:07:47.959085+0000 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: min_mon_release 19 (squid) 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: election_strategy: 1 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-26T21:09:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-26T21:09:57.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: fsmap 2024-04-26T21:09:57.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:09:57.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: mgrmap e13: x(active, since 61s) 2024-04-26T21:09:57.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: overall HEALTH_OK 2024-04-26T21:09:57.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:57.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:57.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:57.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.914+0000 7f777fa79700 1 -- 172.21.15.129:0/2000258809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7778103ff0 msgr2=0x7f77781043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:57.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.914+0000 7f777fa79700 1 --2- 172.21.15.129:0/2000258809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7778103ff0 0x7f77781043d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7774009910 tx=0x7f7774009c20 comp rx=0 tx=0).stop 2024-04-26T21:09:57.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.915+0000 7f777fa79700 1 -- 172.21.15.129:0/2000258809 shutdown_connections 2024-04-26T21:09:57.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.915+0000 7f777fa79700 1 --2- 172.21.15.129:0/2000258809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7778103ff0 0x7f77781043d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:57.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.915+0000 7f777fa79700 1 -- 172.21.15.129:0/2000258809 >> 172.21.15.129:0/2000258809 conn(0x7f77780fd860 msgr2=0x7f77780ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.915+0000 7f777fa79700 1 -- 172.21.15.129:0/2000258809 shutdown_connections 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.915+0000 7f777fa79700 1 -- 172.21.15.129:0/2000258809 wait complete. 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.915+0000 7f777fa79700 1 Processor -- start 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.915+0000 7f777fa79700 1 -- start start 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777fa79700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778103ff0 0x7f7778101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777fa79700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7778102190 0x7f7778102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777fa79700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77781a4d10 0x7f77781a7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777fa79700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f777810d390 con 0x7f77781a4d10 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777fa79700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f777810d210 con 0x7f7778103ff0 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777fa79700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f777810d510 con 0x7f7778102190 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777e016700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77781a4d10 0x7f77781a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777e016700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77781a4d10 0x7f77781a7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:32910/0 (socket says 172.21.15.129:32910) 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777e016700 1 -- 172.21.15.129:0/2837995268 learned_addr learned my addr 172.21.15.129:0/2837995268 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:09:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777d815700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778103ff0 0x7f7778101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777d014700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7778102190 0x7f7778102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777e016700 1 -- 172.21.15.129:0/2837995268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7778102190 msgr2=0x7f7778102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777e016700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7778102190 0x7f7778102610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777e016700 1 -- 172.21.15.129:0/2837995268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778103ff0 msgr2=0x7f7778101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777e016700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778103ff0 0x7f7778101c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777e016700 1 -- 172.21.15.129:0/2837995268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77781a7830 con 0x7f77781a4d10 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777d014700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7778102190 0x7f7778102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.916+0000 7f777e016700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77781a4d10 0x7f77781a7100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f777000ba50 tx=0x7f777000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.917+0000 7f776effd700 1 -- 172.21.15.129:0/2837995268 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770019070 con 0x7f77781a4d10 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.917+0000 7f776effd700 1 -- 172.21.15.129:0/2837995268 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7770014440 con 0x7f77781a4d10 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.917+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77781a7b20 con 0x7f77781a4d10 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.917+0000 7f776effd700 1 -- 172.21.15.129:0/2837995268 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777000ce60 con 0x7f77781a4d10 2024-04-26T21:09:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.917+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781a8060 con 0x7f77781a4d10 2024-04-26T21:09:57.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.918+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7778064a80 con 0x7f77781a4d10 2024-04-26T21:09:57.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.918+0000 7f776effd700 1 -- 172.21.15.129:0/2837995268 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f777001c070 con 0x7f77781a4d10 2024-04-26T21:09:57.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.918+0000 7f776effd700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7764039ae0 0x7f776403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.918+0000 7f776effd700 1 -- 172.21.15.129:0/2837995268 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f777004bdf0 con 0x7f77781a4d10 2024-04-26T21:09:57.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.918+0000 7f777d815700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7764039ae0 0x7f776403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:57.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.918+0000 7f777d815700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7764039ae0 0x7f776403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7774006010 tx=0x7f777401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:57.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:57.923+0000 7f776effd700 1 -- 172.21.15.129:0/2837995268 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7770012500 con 0x7f77781a4d10 2024-04-26T21:09:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.136+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f77781a8300 con 0x7f77781a4d10 2024-04-26T21:09:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.137+0000 7f776effd700 1 -- 172.21.15.129:0/2837995268 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f777000e090 con 0x7f77781a4d10 2024-04-26T21:09:58.138 INFO:teuthology.orchestra.run.smithi129.stdout:# minimal ceph.conf for eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:58.138 INFO:teuthology.orchestra.run.smithi129.stdout:[global] 2024-04-26T21:09:58.139 INFO:teuthology.orchestra.run.smithi129.stdout: fsid = eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:09:58.139 INFO:teuthology.orchestra.run.smithi129.stdout: mon_host = [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.139+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7764039ae0 msgr2=0x7f776403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.139+0000 7f777fa79700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7764039ae0 0x7f776403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7774006010 tx=0x7f777401a040 comp rx=0 tx=0).stop 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.139+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77781a4d10 msgr2=0x7f77781a7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.139+0000 7f777fa79700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77781a4d10 0x7f77781a7100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f777000ba50 tx=0x7f777000bd60 comp rx=0 tx=0).stop 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.139+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 shutdown_connections 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.140+0000 7f777fa79700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7764039ae0 0x7f776403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.140+0000 7f777fa79700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77781a4d10 0x7f77781a7100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.140+0000 7f777fa79700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778103ff0 0x7f7778101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.140+0000 7f777fa79700 1 --2- 172.21.15.129:0/2837995268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7778102190 0x7f7778102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.140+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 >> 172.21.15.129:0/2837995268 conn(0x7f77780fd860 msgr2=0x7f7778106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.140+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 shutdown_connections 2024-04-26T21:09:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:09:58.140+0000 7f777fa79700 1 -- 172.21.15.129:0/2837995268 wait complete. 2024-04-26T21:09:58.428 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-26T21:09:58.428 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-26T21:09:58.428 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-26T21:09:58.428 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:58.428 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: Updating smithi129:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:58.428 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: Updating smithi150:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:58.428 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: Updating smithi171:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:58.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:58.430 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:58 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2837995268' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-26T21:09:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-26T21:09:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-26T21:09:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: Updating smithi129:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: Updating smithi150:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: Updating smithi171:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:58.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:58 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2837995268' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: Updating smithi129:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: Updating smithi150:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: Updating smithi171:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:09:58.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:58 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2837995268' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:59.598 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:59 smithi129 ceph-mon[29237]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:09:59.599 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:09:59 smithi129 ceph-mon[29237]: Reconfiguring daemon mon.a on smithi129 2024-04-26T21:09:59.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:59 smithi150 ceph-mon[32290]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:09:59.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:09:59 smithi150 ceph-mon[32290]: Reconfiguring daemon mon.a on smithi129 2024-04-26T21:09:59.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:59 smithi171 ceph-mon[33478]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:09:59.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:09:59 smithi171 ceph-mon[33478]: Reconfiguring daemon mon.a on smithi129 2024-04-26T21:09:59.932 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T21:09:59.933 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:09:59.933 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:09:59.959 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:09:59.959 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:10:00.024 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:10:00.024 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:10:00.066 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:10:00.066 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:10:00.130 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:10:00.130 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:10:00.161 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:10:00.161 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:10:00.229 INFO:tasks.cephadm:Adding mgr.x on smithi129 2024-04-26T21:10:00.230 INFO:tasks.cephadm:Adding mgr.y on smithi150 2024-04-26T21:10:00.230 INFO:tasks.cephadm:Adding mgr.z on smithi171 2024-04-26T21:10:00.230 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi129=x;smithi150=y;smithi171=z' 2024-04-26T21:10:00.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:00 smithi171 ceph-mon[33478]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:10:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:10:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:00 smithi171 ceph-mon[33478]: overall HEALTH_OK 2024-04-26T21:10:00.503 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:10:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:00 smithi150 ceph-mon[32290]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:10:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:10:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:00 smithi150 ceph-mon[32290]: overall HEALTH_OK 2024-04-26T21:10:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:00 smithi129 ceph-mon[29237]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:10:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:10:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:00 smithi129 ceph-mon[29237]: overall HEALTH_OK 2024-04-26T21:10:01.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:01 smithi150 ceph-mon[32290]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:10:01.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:01 smithi150 ceph-mon[32290]: Reconfiguring daemon mon.b on smithi150 2024-04-26T21:10:01.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:01.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:01.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:10:01.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:10:01.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:01.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:01 smithi129 ceph-mon[29237]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:10:01.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:01 smithi129 ceph-mon[29237]: Reconfiguring daemon mon.b on smithi150 2024-04-26T21:10:01.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:01.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:01.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:10:01.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:10:01.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:01.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:01 smithi171 ceph-mon[33478]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:10:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:01 smithi171 ceph-mon[33478]: Reconfiguring daemon mon.b on smithi150 2024-04-26T21:10:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:10:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:10:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:02.068 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.065+0000 7f522aef4700 1 -- 172.21.15.171:0/3904347189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 msgr2=0x7f52241033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.068 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.065+0000 7f522aef4700 1 --2- 172.21.15.171:0/3904347189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 0x7f52241033d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f521c00b0a0 tx=0x7f521c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.068 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.065+0000 7f522aef4700 1 -- 172.21.15.171:0/3904347189 shutdown_connections 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.065+0000 7f522aef4700 1 --2- 172.21.15.171:0/3904347189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 0x7f52241033d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.065+0000 7f522aef4700 1 -- 172.21.15.171:0/3904347189 >> 172.21.15.171:0/3904347189 conn(0x7f52240fe550 msgr2=0x7f5224100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.065+0000 7f522aef4700 1 -- 172.21.15.171:0/3904347189 shutdown_connections 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.065+0000 7f522aef4700 1 -- 172.21.15.171:0/3904347189 wait complete. 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.065+0000 7f522aef4700 1 Processor -- start 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.065+0000 7f522aef4700 1 -- start start 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f522aef4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 0x7f52241986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f522aef4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5224198c30 0x7f522419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f522aef4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52241990b0 0x7f52241a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f522aef4700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5224108ea0 con 0x7f5224102fb0 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f522aef4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5224108d20 con 0x7f52241990b0 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f522aef4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5224109020 con 0x7f5224198c30 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5223fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5224198c30 0x7f522419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5228c90700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 0x7f52241986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5228c90700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 0x7f52241986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.171:56128/0 (socket says 172.21.15.171:56128) 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5228c90700 1 -- 172.21.15.171:0/1322704190 learned_addr learned my addr 172.21.15.171:0/1322704190 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5229491700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52241990b0 0x7f52241a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5228c90700 1 -- 172.21.15.171:0/1322704190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5224198c30 msgr2=0x7f522419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5228c90700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5224198c30 0x7f522419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5228c90700 1 -- 172.21.15.171:0/1322704190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52241990b0 msgr2=0x7f52241a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5228c90700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52241990b0 0x7f52241a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5228c90700 1 -- 172.21.15.171:0/1322704190 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52241a0b00 con 0x7f5224102fb0 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5228c90700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 0x7f52241986f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f521c015040 tx=0x7f521c00bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5221ffb700 1 -- 172.21.15.171:0/1322704190 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c017040 con 0x7f5224102fb0 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f5221ffb700 1 -- 172.21.15.171:0/1322704190 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f521c005780 con 0x7f5224102fb0 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.066+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52241a0d90 con 0x7f5224102fb0 2024-04-26T21:10:02.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.067+0000 7f5221ffb700 1 -- 172.21.15.171:0/1322704190 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c012a80 con 0x7f5224102fb0 2024-04-26T21:10:02.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.067+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52241a1290 con 0x7f5224102fb0 2024-04-26T21:10:02.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.068+0000 7f5221ffb700 1 -- 172.21.15.171:0/1322704190 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f521c012c20 con 0x7f5224102fb0 2024-04-26T21:10:02.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.068+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5224064a80 con 0x7f5224102fb0 2024-04-26T21:10:02.072 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.068+0000 7f5221ffb700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5210039ae0 0x7f521003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.072 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.068+0000 7f5221ffb700 1 -- 172.21.15.171:0/1322704190 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f521c00f020 con 0x7f5224102fb0 2024-04-26T21:10:02.072 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.068+0000 7f5223fff700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5210039ae0 0x7f521003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.072 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.069+0000 7f5223fff700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5210039ae0 0x7f521003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5214009ad0 tx=0x7f5214009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.075 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.073+0000 7f5221ffb700 1 -- 172.21.15.171:0/1322704190 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f521c029020 con 0x7f5224102fb0 2024-04-26T21:10:02.291 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.290+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi129=x;smithi150=y;smithi171=z", "target": ["mon-mgr", ""]}) -- 0x7f5224063860 con 0x7f5210039ae0 2024-04-26T21:10:02.298 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.296+0000 7f5221ffb700 1 -- 172.21.15.171:0/1322704190 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5224063860 con 0x7f5210039ae0 2024-04-26T21:10:02.298 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mgr update... 2024-04-26T21:10:02.300 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.298+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5210039ae0 msgr2=0x7f521003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.298+0000 7f522aef4700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5210039ae0 0x7f521003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5214009ad0 tx=0x7f5214009210 comp rx=0 tx=0).stop 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.298+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 msgr2=0x7f52241986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.298+0000 7f522aef4700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 0x7f52241986f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f521c015040 tx=0x7f521c00bd10 comp rx=0 tx=0).stop 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.299+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 shutdown_connections 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.299+0000 7f522aef4700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5210039ae0 0x7f521003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.299+0000 7f522aef4700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52241990b0 0x7f52241a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.299+0000 7f522aef4700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5224102fb0 0x7f52241986f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.299+0000 7f522aef4700 1 --2- 172.21.15.171:0/1322704190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5224198c30 0x7f522419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.299+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 >> 172.21.15.171:0/1322704190 conn(0x7f52240fe550 msgr2=0x7f52240ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.299+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 shutdown_connections 2024-04-26T21:10:02.301 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:02.299+0000 7f522aef4700 1 -- 172.21.15.171:0/1322704190 wait complete. 2024-04-26T21:10:02.596 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:02 smithi171 ceph-mon[33478]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:10:02.596 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:02 smithi171 ceph-mon[33478]: Reconfiguring daemon mon.c on smithi171 2024-04-26T21:10:02.596 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:02 smithi171 ceph-mon[33478]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:02.596 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:02 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:02 smithi150 ceph-mon[32290]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:10:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:02 smithi150 ceph-mon[32290]: Reconfiguring daemon mon.c on smithi171 2024-04-26T21:10:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:02 smithi150 ceph-mon[32290]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:02 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:02.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:02 smithi129 ceph-mon[29237]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:10:02.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:02 smithi129 ceph-mon[29237]: Reconfiguring daemon mon.c on smithi171 2024-04-26T21:10:02.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:02 smithi129 ceph-mon[29237]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:02.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:02 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:03.007 DEBUG:teuthology.orchestra.run.smithi150:mgr.y> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T21:10:03.010 DEBUG:teuthology.orchestra.run.smithi171:mgr.z> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T21:10:03.012 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T21:10:03.012 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:10:03.012 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:10:03.032 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:10:03.032 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-04-26T21:10:03.090 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:10:03.090 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:03.090 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T21:10:03.090 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:03.090 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:03.090 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-26 21:09:08.310886561 +0000 2024-04-26T21:10:03.090 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-26 21:03:21.321367883 +0000 2024-04-26T21:10:03.090 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-26 21:03:21.321367883 +0000 2024-04-26T21:10:03.090 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-26 21:03:21.321367883 +0000 2024-04-26T21:10:03.091 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:10:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-26T21:10:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-26T21:10:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000263444 s, 1.9 MB/s 2024-04-26T21:10:03.158 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:10:03.217 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-04-26T21:10:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:10:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:03.275 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T21:10:03.275 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:03.275 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:03.275 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-26 21:09:08.347885759 +0000 2024-04-26T21:10:03.275 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-26 21:03:21.662360102 +0000 2024-04-26T21:10:03.275 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-26 21:03:21.662360102 +0000 2024-04-26T21:10:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-26 21:03:21.662360102 +0000 2024-04-26T21:10:03.275 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:10:03.340 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-26T21:10:03.340 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-26T21:10:03.340 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000201008 s, 2.5 MB/s 2024-04-26T21:10:03.341 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:10:03.399 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-04-26T21:10:03.455 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:10:03.455 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T21:10:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-26 21:09:08.382885000 +0000 2024-04-26T21:10:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-26 21:03:21.997352458 +0000 2024-04-26T21:10:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-26 21:03:21.997352458 +0000 2024-04-26T21:10:03.456 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-26 21:03:21.997352458 +0000 2024-04-26T21:10:03.456 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:10:03.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:03 smithi129 ceph-mon[29237]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi129=x;smithi150=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:10:03.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:03 smithi129 ceph-mon[29237]: Saving service mgr spec with placement smithi129=x;smithi150=y;smithi171=z;count:3 2024-04-26T21:10:03.494 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-26T21:10:03.494 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-26T21:10:03.494 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000224067 s, 2.3 MB/s 2024-04-26T21:10:03.495 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:10:03.553 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-04-26T21:10:03.609 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:10:03.609 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:03.609 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T21:10:03.609 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-26 21:09:08.417884241 +0000 2024-04-26T21:10:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-26 21:03:22.337344700 +0000 2024-04-26T21:10:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-26 21:03:22.337344700 +0000 2024-04-26T21:10:03.610 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-26 21:03:22.337344700 +0000 2024-04-26T21:10:03.610 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:10:03.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:03 smithi150 ceph-mon[32290]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi129=x;smithi150=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:10:03.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:03 smithi150 ceph-mon[32290]: Saving service mgr spec with placement smithi129=x;smithi150=y;smithi171=z;count:3 2024-04-26T21:10:03.674 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-26T21:10:03.674 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-26T21:10:03.674 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000194516 s, 2.6 MB/s 2024-04-26T21:10:03.675 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:10:03.731 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-26T21:10:03.731 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:10:03.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:03 smithi171 ceph-mon[33478]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi129=x;smithi150=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:10:03.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:03 smithi171 ceph-mon[33478]: Saving service mgr spec with placement smithi129=x;smithi150=y;smithi171=z;count:3 2024-04-26T21:10:03.750 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:10:03.750 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-04-26T21:10:03.808 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:10:03.808 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:03.808 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T21:10:03.808 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:03.808 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:03.809 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-26 21:09:49.197277453 +0000 2024-04-26T21:10:03.809 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-26 21:04:51.970396120 +0000 2024-04-26T21:10:03.809 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-26 21:04:51.970396120 +0000 2024-04-26T21:10:03.809 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-26 21:04:51.970396120 +0000 2024-04-26T21:10:03.809 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:10:03.874 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-26T21:10:03.875 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-26T21:10:03.875 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000289272 s, 1.8 MB/s 2024-04-26T21:10:03.876 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:10:03.933 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-04-26T21:10:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:10:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:03.990 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T21:10:03.990 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:03.990 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:03.990 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-26 21:09:49.198277433 +0000 2024-04-26T21:10:03.990 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-26 21:04:52.308389150 +0000 2024-04-26T21:10:03.990 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-26 21:04:52.308389150 +0000 2024-04-26T21:10:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-26 21:04:52.308389150 +0000 2024-04-26T21:10:03.991 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:10:04.054 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-26T21:10:04.054 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-26T21:10:04.054 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000218376 s, 2.3 MB/s 2024-04-26T21:10:04.055 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:10:04.112 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-04-26T21:10:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:10:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:04.168 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T21:10:04.168 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:04.168 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:04.168 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-26 21:09:49.198277433 +0000 2024-04-26T21:10:04.168 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-26 21:04:52.651382077 +0000 2024-04-26T21:10:04.169 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-26 21:04:52.651382077 +0000 2024-04-26T21:10:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-26 21:04:52.651382077 +0000 2024-04-26T21:10:04.169 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:10:04.231 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-26T21:10:04.231 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-26T21:10:04.231 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000200279 s, 2.6 MB/s 2024-04-26T21:10:04.232 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:10:04.289 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-04-26T21:10:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:10:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:04.344 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T21:10:04.344 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:04.344 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:04.344 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-26 21:09:49.198277433 +0000 2024-04-26T21:10:04.344 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-26 21:04:52.991375066 +0000 2024-04-26T21:10:04.344 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-26 21:04:52.991375066 +0000 2024-04-26T21:10:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-26 21:04:52.991375066 +0000 2024-04-26T21:10:04.345 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:10:04.406 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-26T21:10:04.406 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-26T21:10:04.406 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000197098 s, 2.6 MB/s 2024-04-26T21:10:04.407 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:10:04.463 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-26T21:10:04.463 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:10:04.478 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:10:04.479 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-04-26T21:10:04.533 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:10:04.533 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:04.533 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T21:10:04.533 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:04.533 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:04.534 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-26 21:09:39.831371186 +0000 2024-04-26T21:10:04.534 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-26 21:04:06.037676605 +0000 2024-04-26T21:10:04.534 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-26 21:04:06.037676605 +0000 2024-04-26T21:10:04.534 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-26 21:04:06.037676605 +0000 2024-04-26T21:10:04.534 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:10:04.594 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:04.594 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:04.594 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:04.594 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:04.594 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:04.594 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:04.594 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:04.595 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:10:04.595 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:10:04.595 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:10:04.595 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:04.595 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-26T21:10:04.596 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-26T21:10:04.596 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.00026688 s, 1.9 MB/s 2024-04-26T21:10:04.597 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:10:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:10:04.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:04.654 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-04-26T21:10:04.710 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:10:04.710 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:04.710 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T21:10:04.710 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:04.710 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:04.710 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-26 21:09:39.831371186 +0000 2024-04-26T21:10:04.710 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-26 21:04:06.367669881 +0000 2024-04-26T21:10:04.710 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-26 21:04:06.367669881 +0000 2024-04-26T21:10:04.710 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-26 21:04:06.367669881 +0000 2024-04-26T21:10:04.710 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:10:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:04.773 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-26T21:10:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-26T21:10:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000199698 s, 2.6 MB/s 2024-04-26T21:10:04.775 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:10:04.830 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-04-26T21:10:04.885 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:10:04.885 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:04.885 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T21:10:04.885 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:04.885 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:04.885 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-26 21:09:39.831371186 +0000 2024-04-26T21:10:04.885 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-26 21:04:06.705662995 +0000 2024-04-26T21:10:04.885 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-26 21:04:06.705662995 +0000 2024-04-26T21:10:04.885 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-26 21:04:06.705662995 +0000 2024-04-26T21:10:04.885 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:10:04.947 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-26T21:10:04.947 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-26T21:10:04.947 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000171142 s, 3.0 MB/s 2024-04-26T21:10:04.948 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:10:05.004 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-04-26T21:10:05.057 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:10:05.058 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:10:05.058 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T21:10:05.058 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:10:05.058 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:10:05.058 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-26 21:09:39.832371166 +0000 2024-04-26T21:10:05.058 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-26 21:04:07.042656129 +0000 2024-04-26T21:10:05.058 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-26 21:04:07.042656129 +0000 2024-04-26T21:10:05.058 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-26 21:04:07.042656129 +0000 2024-04-26T21:10:05.058 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:10:05.124 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-26T21:10:05.125 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-26T21:10:05.125 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000169042 s, 3.0 MB/s 2024-04-26T21:10:05.126 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:10:05.182 INFO:tasks.cephadm:Deploying osd.0 on smithi129 with /dev/vg_nvme/lv_4... 2024-04-26T21:10:05.183 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:10:05.425 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:05.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:05 smithi129 ceph-mon[29237]: Deploying daemon mgr.z on smithi171 2024-04-26T21:10:05.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:05 smithi171 ceph-mon[33478]: Deploying daemon mgr.z on smithi171 2024-04-26T21:10:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:05 smithi150 ceph-mon[32290]: Deploying daemon mgr.z on smithi171 2024-04-26T21:10:06.315 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 systemd[1]: Starting Ceph mgr.z for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:10:06.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:06 smithi129 ceph-mon[29237]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:06 smithi150 ceph-mon[32290]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:06.639 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:06 smithi171 ceph-mon[33478]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:06.640 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 podman[35326]: 2024-04-26 21:10:06.42867674 +0000 UTC m=+0.115041097 container create 22238896e9156177836bb3e1566f925519f5cc715368ad816dcc8a6936d8add5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T21:10:06.640 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 podman[35326]: 2024-04-26 21:10:06.332308088 +0000 UTC m=+0.018672412 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:10:06.640 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 podman[35326]: 2024-04-26 21:10:06.639649578 +0000 UTC m=+0.326013889 container init 22238896e9156177836bb3e1566f925519f5cc715368ad816dcc8a6936d8add5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T21:10:06.980 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 podman[35326]: 2024-04-26 21:10:06.642737421 +0000 UTC m=+0.329101733 container start 22238896e9156177836bb3e1566f925519f5cc715368ad816dcc8a6936d8add5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T21:10:06.981 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 ceph-mgr[35346]: -- 172.21.15.171:0/2076898097 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563eed4ae5a0 con 0x563eed482800 2024-04-26T21:10:06.981 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 bash[35326]: 22238896e9156177836bb3e1566f925519f5cc715368ad816dcc8a6936d8add5 2024-04-26T21:10:06.981 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 systemd[1]: Started Ceph mgr.z for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:10:06.981 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:06.794+0000 7fadd42a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:10:06.981 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:06.978+0000 7fadd42a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:10:07.241 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:07.080+0000 7fadd42a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:10:07.293 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:10:07.294 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:10:07.294 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-26T21:10:07.294 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-26T21:10:07.294 INFO:teuthology.orchestra.run.smithi129.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125089 s, 838 MB/s 2024-04-26T21:10:07.294 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 2024-04-26T21:10:07.294 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-26T21:10:07.741 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:07.389+0000 7fadd42a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:10:07.741 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:07.473+0000 7fadd42a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:10:07.960 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:07.960 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:07.960 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:07.960 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:10:07.960 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:10:07.960 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:10:07.960 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:07.961 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:07 smithi129 ceph-mon[29237]: Deploying daemon mgr.y on smithi150 2024-04-26T21:10:07.961 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:07 smithi129 ceph-mon[29237]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:07.987 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_4 2024-04-26T21:10:08.086 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:08.086 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:08.086 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:08.086 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:10:08.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:10:08.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:10:08.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:08.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-mon[33478]: Deploying daemon mgr.y on smithi150 2024-04-26T21:10:08.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:07 smithi171 ceph-mon[33478]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:08.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:08.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:08.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:08.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:10:08.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:10:08.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:10:08.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:08.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:07 smithi150 ceph-mon[32290]: Deploying daemon mgr.y on smithi150 2024-04-26T21:10:08.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:07 smithi150 ceph-mon[32290]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:08.220 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:08.991 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:08.707+0000 7fadd42a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:10:08.991 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:08.881+0000 7fadd42a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:10:09.347 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:09.058+0000 7fadd42a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:10:09.347 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:09.345+0000 7fadd42a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:10:09.633 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:09 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:09.523+0000 7fb84d086200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:10:09.647 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:09.438+0000 7fadd42a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:10:09.647 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:09.645+0000 7fadd42a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:10:09.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.771+0000 7f04a7a81700 1 -- 172.21.15.129:0/849372215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0103480 msgr2=0x7f04a01038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.771+0000 7f04a7a81700 1 --2- 172.21.15.129:0/849372215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0103480 0x7f04a01038a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f04980099c0 tx=0x7f0498009cd0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.771+0000 7f04a7a81700 1 -- 172.21.15.129:0/849372215 shutdown_connections 2024-04-26T21:10:09.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.771+0000 7f04a7a81700 1 --2- 172.21.15.129:0/849372215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04a0105020 0x7f04a010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.771+0000 7f04a7a81700 1 --2- 172.21.15.129:0/849372215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0103480 0x7f04a01038a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.771+0000 7f04a7a81700 1 --2- 172.21.15.129:0/849372215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04a0104680 0x7f04a0104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.771+0000 7f04a7a81700 1 -- 172.21.15.129:0/849372215 >> 172.21.15.129:0/849372215 conn(0x7f04a00fea20 msgr2=0x7f04a0100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.771+0000 7f04a7a81700 1 -- 172.21.15.129:0/849372215 shutdown_connections 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.771+0000 7f04a7a81700 1 -- 172.21.15.129:0/849372215 wait complete. 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a7a81700 1 Processor -- start 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a7a81700 1 -- start start 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a7a81700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04a0103480 0x7f04a01995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a7a81700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04a0104680 0x7f04a0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a7a81700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0105020 0x7f04a01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a7a81700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010ef20 con 0x7f04a0105020 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a7a81700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010eda0 con 0x7f04a0104680 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a7a81700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010f0a0 con 0x7f04a0103480 2024-04-26T21:10:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a601e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0105020 0x7f04a01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a601e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0105020 0x7f04a01a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58254/0 (socket says 172.21.15.129:58254) 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a581d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04a0103480 0x7f04a01995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a601e700 1 -- 172.21.15.129:0/3484358401 learned_addr learned my addr 172.21.15.129:0/3484358401 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a501c700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04a0104680 0x7f04a0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a601e700 1 -- 172.21.15.129:0/3484358401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04a0103480 msgr2=0x7f04a01995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a601e700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04a0103480 0x7f04a01995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a601e700 1 -- 172.21.15.129:0/3484358401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04a0104680 msgr2=0x7f04a0199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a601e700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04a0104680 0x7f04a0199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.772+0000 7f04a601e700 1 -- 172.21.15.129:0/3484358401 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a01a1250 con 0x7f04a0105020 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.773+0000 7f04a601e700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0105020 0x7f04a01a0bb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f049c00ef60 tx=0x7f049c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.773+0000 7f0496ffd700 1 -- 172.21.15.129:0/3484358401 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c0092e0 con 0x7f04a0105020 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.773+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a0075650 con 0x7f04a0105020 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.773+0000 7f0496ffd700 1 -- 172.21.15.129:0/3484358401 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f049c004bb0 con 0x7f04a0105020 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.773+0000 7f0496ffd700 1 -- 172.21.15.129:0/3484358401 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c004020 con 0x7f04a0105020 2024-04-26T21:10:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.773+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a0075b40 con 0x7f04a0105020 2024-04-26T21:10:09.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.774+0000 7f0496ffd700 1 -- 172.21.15.129:0/3484358401 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f049c01b030 con 0x7f04a0105020 2024-04-26T21:10:09.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.774+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a006a110 con 0x7f04a0105020 2024-04-26T21:10:09.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.774+0000 7f0496ffd700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f048c039890 0x7f048c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.774+0000 7f0496ffd700 1 -- 172.21.15.129:0/3484358401 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f049c004d60 con 0x7f04a0105020 2024-04-26T21:10:09.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.774+0000 7f04a581d700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f048c039890 0x7f048c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.774+0000 7f04a581d700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f048c039890 0x7f048c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f049800b580 tx=0x7f0498005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.779+0000 7f0496ffd700 1 -- 172.21.15.129:0/3484358401 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f049c022020 con 0x7f04a0105020 2024-04-26T21:10:09.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:09.987+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f04a0063860 con 0x7f048c039890 2024-04-26T21:10:09.991 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:09.849+0000 7fadd42a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:10:10.133 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:09 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:09.711+0000 7fb84d086200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:10:10.133 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:09 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:09.820+0000 7fb84d086200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:10:10.431 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:10.140+0000 7fb84d086200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:10.236+0000 7fb84d086200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:10:10.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:10.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.596 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:10.596 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.597 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.597 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.597 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.597 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:10.597 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:10:10.597 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:10:10.597 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:10.597 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.631 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:10.488+0000 7fadd42a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:10.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.241 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:10.922+0000 7fadd42a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:10:11.241 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:11.031+0000 7fadd42a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:10:11.241 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:11.239+0000 7fadd42a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:10:11.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:10:11.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:10:11.592 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:11.623 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:11.456+0000 7fb84d086200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:10:11.623 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:11.622+0000 7fb84d086200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:10:11.623 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:10:11.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:11.883 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:11 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:11.792+0000 7fb84d086200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:10:12.099 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:11.762+0000 7fadd42a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:10:12.099 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:11.854+0000 7fadd42a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:10:12.366 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:12 smithi150 ceph-mon[32290]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:10:12.366 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:12 smithi150 ceph-mon[32290]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:12.366 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:12 smithi150 ceph-mon[32290]: Reconfiguring daemon mgr.x on smithi129 2024-04-26T21:10:12.367 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:12 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:12.073+0000 7fb84d086200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:10:12.367 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:12 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:12.163+0000 7fb84d086200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:10:12.367 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:12 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:12.365+0000 7fb84d086200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:10:12.442 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:12 smithi171 ceph-mon[33478]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:10:12.442 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:12 smithi171 ceph-mon[33478]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:12.442 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:12 smithi171 ceph-mon[33478]: Reconfiguring daemon mgr.x on smithi129 2024-04-26T21:10:12.442 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:12.097+0000 7fadd42a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:10:12.442 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:12.227+0000 7fadd42a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:10:12.443 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:12.324+0000 7fadd42a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:10:12.443 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:12.440+0000 7fadd42a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:10:12.632 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:12 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:12.554+0000 7fb84d086200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:10:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:12 smithi129 ceph-mon[29237]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:10:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:12 smithi129 ceph-mon[29237]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:12 smithi129 ceph-mon[29237]: Reconfiguring daemon mgr.x on smithi129 2024-04-26T21:10:13.241 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:12.753+0000 7fadd42a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:10:13.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:13 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2755320233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03a78174-76a0-4fbb-baf7-222c97cf105c"}]: dispatch 2024-04-26T21:10:13.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:13 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2755320233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03a78174-76a0-4fbb-baf7-222c97cf105c"}]': finished 2024-04-26T21:10:13.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:13 smithi129 ceph-mon[29237]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:10:13.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:13 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:13.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:13 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2755320233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03a78174-76a0-4fbb-baf7-222c97cf105c"}]: dispatch 2024-04-26T21:10:13.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:13 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2755320233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03a78174-76a0-4fbb-baf7-222c97cf105c"}]': finished 2024-04-26T21:10:13.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:13 smithi150 ceph-mon[32290]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:10:13.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:13 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:13.563 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:13 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:13.150+0000 7fb84d086200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:10:13.569 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:13 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2755320233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03a78174-76a0-4fbb-baf7-222c97cf105c"}]: dispatch 2024-04-26T21:10:13.569 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:13 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2755320233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03a78174-76a0-4fbb-baf7-222c97cf105c"}]': finished 2024-04-26T21:10:13.569 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:13 smithi171 ceph-mon[33478]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:10:13.570 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:13 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:13.570 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:13.354+0000 7fadd42a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:10:13.570 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:10:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z[35342]: 2024-04-26T21:10:13.567+0000 7fadd42a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:10:13.854 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:13 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:13.561+0000 7fb84d086200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:10:13.854 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:13 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:13.656+0000 7fb84d086200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:10:13.854 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:13 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:13.853+0000 7fb84d086200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:10:14.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:14.251+0000 7f0496ffd700 1 -- 172.21.15.129:0/3484358401 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 14) ==== 92603+0+0 (secure 0 0 0) 0x7f049c012050 con 0x7f04a0105020 2024-04-26T21:10:14.633 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:14.370+0000 7fb84d086200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:10:14.633 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:14.459+0000 7fb84d086200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:10:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-mon[32290]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/996668073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:10:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-mon[32290]: Standby manager daemon z started 2024-04-26T21:10:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-mon[32290]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:10:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-mon[32290]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:10:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-mon[32290]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:10:14.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-mon[32290]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:10:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:14 smithi129 ceph-mon[29237]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:14 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/996668073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:10:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:14 smithi129 ceph-mon[29237]: Standby manager daemon z started 2024-04-26T21:10:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:14 smithi129 ceph-mon[29237]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:10:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:14 smithi129 ceph-mon[29237]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:10:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:14 smithi129 ceph-mon[29237]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:10:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:14 smithi129 ceph-mon[29237]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:10:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:14 smithi171 ceph-mon[33478]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:14 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/996668073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:10:14.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:14 smithi171 ceph-mon[33478]: Standby manager daemon z started 2024-04-26T21:10:14.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:14 smithi171 ceph-mon[33478]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:10:14.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:14 smithi171 ceph-mon[33478]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:10:14.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:14 smithi171 ceph-mon[33478]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:10:14.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:14 smithi171 ceph-mon[33478]: from='mgr.? 172.21.15.171:0/1037493178' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:10:15.030 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:14.696+0000 7fb84d086200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:10:15.030 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:14.825+0000 7fb84d086200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:10:15.030 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:14 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:14.916+0000 7fb84d086200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:10:15.030 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:15 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:15.028+0000 7fb84d086200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:10:15.330 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:15 smithi150 ceph-mon[32290]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-26T21:10:15.330 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:10:15.633 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:15 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:15.329+0000 7fb84d086200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:10:15.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:15 smithi129 ceph-mon[29237]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-26T21:10:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:10:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:15 smithi171 ceph-mon[33478]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-26T21:10:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:10:16.262 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:15 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:15.917+0000 7fb84d086200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:10:16.262 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:10:16 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:10:16.127+0000 7fb84d086200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:10:16.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:16.271+0000 7f0496ffd700 1 -- 172.21.15.129:0/3484358401 <== mon.0 v2:172.21.15.129:3300/0 8 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f049c019d30 con 0x7f04a0105020 2024-04-26T21:10:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:16 smithi150 ceph-mon[32290]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:16 smithi150 ceph-mon[32290]: Standby manager daemon y started 2024-04-26T21:10:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:16 smithi150 ceph-mon[32290]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:10:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:16 smithi150 ceph-mon[32290]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:10:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:16 smithi150 ceph-mon[32290]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:10:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:16 smithi150 ceph-mon[32290]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:10:16.645 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:16 smithi129 ceph-mon[29237]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:16.645 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:16 smithi129 ceph-mon[29237]: Standby manager daemon y started 2024-04-26T21:10:16.645 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:16 smithi129 ceph-mon[29237]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:10:16.646 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:16 smithi129 ceph-mon[29237]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:10:16.646 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:16 smithi129 ceph-mon[29237]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:10:16.646 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:16 smithi129 ceph-mon[29237]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:10:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:16 smithi171 ceph-mon[33478]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:16 smithi171 ceph-mon[33478]: Standby manager daemon y started 2024-04-26T21:10:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:16 smithi171 ceph-mon[33478]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:10:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:16 smithi171 ceph-mon[33478]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:10:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:16 smithi171 ceph-mon[33478]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:10:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:16 smithi171 ceph-mon[33478]: from='mgr.? 172.21.15.150:0/1011051186' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:10:17.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:17 smithi150 ceph-mon[32290]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-26T21:10:17.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:17 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:10:17.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:17 smithi129 ceph-mon[29237]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-26T21:10:17.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:17 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:10:17.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:17 smithi171 ceph-mon[33478]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-26T21:10:17.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:17 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:10:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:18 smithi129 ceph-mon[29237]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:18.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:18.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:18.536 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:18 smithi171 ceph-mon[33478]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:18.536 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:18.536 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:18.536 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:18.537 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:18.537 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:18.537 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:18 smithi150 ceph-mon[32290]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:20.602 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:20 smithi129 ceph-mon[29237]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:20 smithi150 ceph-mon[32290]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:20 smithi171 ceph-mon[33478]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:21.498 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:10:21.498 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:21.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:10:21.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:21.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:10:21.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:22 smithi129 ceph-mon[29237]: Deploying daemon osd.0 on smithi129 2024-04-26T21:10:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:22 smithi129 ceph-mon[29237]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:22 smithi150 ceph-mon[32290]: Deploying daemon osd.0 on smithi129 2024-04-26T21:10:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:22 smithi150 ceph-mon[32290]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:22.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:22 smithi171 ceph-mon[33478]: Deploying daemon osd.0 on smithi129 2024-04-26T21:10:22.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:22 smithi171 ceph-mon[33478]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:24.568 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:24 smithi129 ceph-mon[29237]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:24.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:24 smithi150 ceph-mon[32290]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:24 smithi171 ceph-mon[33478]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:26 smithi129 ceph-mon[29237]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:26.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:26 smithi150 ceph-mon[32290]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:26.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:26.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:26.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:26 smithi171 ceph-mon[33478]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:26.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:26.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:28.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:28 smithi129 ceph-mon[29237]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:28.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:28 smithi150 ceph-mon[32290]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:28 smithi171 ceph-mon[33478]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:29.699 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 0 on host 'smithi129' 2024-04-26T21:10:29.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.697+0000 7f0496ffd700 1 -- 172.21.15.129:0/3484358401 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f04a0063860 con 0x7f048c039890 2024-04-26T21:10:29.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f048c039890 msgr2=0x7f048c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:29.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f048c039890 0x7f048c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f049800b580 tx=0x7f0498005fb0 comp rx=0 tx=0).stop 2024-04-26T21:10:29.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0105020 msgr2=0x7f04a01a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:29.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0105020 0x7f04a01a0bb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f049c00ef60 tx=0x7f049c00ef90 comp rx=0 tx=0).stop 2024-04-26T21:10:29.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 shutdown_connections 2024-04-26T21:10:29.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f048c039890 0x7f048c03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:29.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04a0105020 0x7f04a01a0bb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:29.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04a0103480 0x7f04a01995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:29.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 --2- 172.21.15.129:0/3484358401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04a0104680 0x7f04a0199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:29.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 >> 172.21.15.129:0/3484358401 conn(0x7f04a00fea20 msgr2=0x7f04a0107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:29.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 shutdown_connections 2024-04-26T21:10:29.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:10:29.700+0000 7f04a7a81700 1 -- 172.21.15.129:0/3484358401 wait complete. 2024-04-26T21:10:30.372 DEBUG:teuthology.orchestra.run.smithi129:osd.0> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T21:10:30.374 INFO:tasks.cephadm:Deploying osd.1 on smithi150 with /dev/vg_nvme/lv_4... 2024-04-26T21:10:30.374 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:10:30.558 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:30 smithi129 ceph-mon[29237]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:30.558 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.558 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.558 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.558 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.558 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:30 smithi129 ceph-mon[29237]: from='osd.0 [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:10:30.558 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:30 smithi129 ceph-mon[29237]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:10:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:10:30.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:30 smithi150 ceph-mon[32290]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:30.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:30 smithi150 ceph-mon[32290]: from='osd.0 [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:10:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:30 smithi150 ceph-mon[32290]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:10:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:30 smithi171 ceph-mon[33478]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:30 smithi171 ceph-mon[33478]: from='osd.0 [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:10:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:30 smithi171 ceph-mon[33478]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:10:31.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:31 smithi150 ceph-mon[32290]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:10:31.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:31 smithi150 ceph-mon[32290]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:10:31.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:31 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:31.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:31 smithi150 ceph-mon[32290]: from='osd.0 [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:10:31.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:31 smithi150 ceph-mon[32290]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:10:31.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:31 smithi150 ceph-mon[32290]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:31.957 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:31 smithi129 ceph-mon[29237]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:10:31.957 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:31 smithi129 ceph-mon[29237]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:10:31.957 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:31 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:31.958 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:31 smithi129 ceph-mon[29237]: from='osd.0 [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:10:31.958 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:31 smithi129 ceph-mon[29237]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:10:31.958 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:31 smithi129 ceph-mon[29237]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:31 smithi171 ceph-mon[33478]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:10:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:31 smithi171 ceph-mon[33478]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:10:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:31 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:31.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:31 smithi171 ceph-mon[33478]: from='osd.0 [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:10:31.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:31 smithi171 ceph-mon[33478]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:10:31.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:31 smithi171 ceph-mon[33478]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:32.522 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:10:32.522 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:10:32.522 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-26T21:10:32.522 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-26T21:10:32.522 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123958 s, 846 MB/s 2024-04-26T21:10:32.522 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-26T21:10:32.720 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:10:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0[44451]: 2024-04-26T21:10:32.233+0000 7feeb7df2700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:10:32.720 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:10:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0[44451]: 2024-04-26T21:10:32.237+0000 7feeafbdd700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:10:32.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:32 smithi171 ceph-mon[33478]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:10:32.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:32 smithi171 ceph-mon[33478]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:10:32.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:32 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:32.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:32 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:32.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:32 smithi171 ceph-mon[33478]: from='osd.0 ' entity='osd.0' 2024-04-26T21:10:33.118 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:32 smithi129 ceph-mon[29237]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:10:33.118 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:32 smithi129 ceph-mon[29237]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:10:33.118 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:32 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:33.118 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:32 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:33.118 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:32 smithi129 ceph-mon[29237]: from='osd.0 ' entity='osd.0' 2024-04-26T21:10:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:32 smithi150 ceph-mon[32290]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:10:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:32 smithi150 ceph-mon[32290]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:10:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:32 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:32 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:32 smithi150 ceph-mon[32290]: from='osd.0 ' entity='osd.0' 2024-04-26T21:10:33.182 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-04-26T21:10:33.421 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:10:33.855 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:10:33.855 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:10:33.855 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:33.855 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:33.855 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: osd.0 [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] boot 2024-04-26T21:10:33.855 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:10:33.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:33.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:33.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:33.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:10:33.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:33.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:33.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:33.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:33.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:34.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:10:34.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:10:34.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:34.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:34.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: osd.0 [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] boot 2024-04-26T21:10:34.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:10:34.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:34.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:34.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:34.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:10:34.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:34.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:34.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:34.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:34.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:34.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: osd.0 [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] boot 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:34.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:34.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:34.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 -- 172.21.15.150:0/316310791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 msgr2=0x7fbeec101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 --2- 172.21.15.150:0/316310791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 0x7fbeec101fe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0099c0 tx=0x7fbedc009cd0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 -- 172.21.15.150:0/316310791 shutdown_connections 2024-04-26T21:10:34.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 --2- 172.21.15.150:0/316310791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec103760 0x7fbeec10a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 --2- 172.21.15.150:0/316310791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 0x7fbeec101fe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 --2- 172.21.15.150:0/316310791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeec102dc0 0x7fbeec103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 -- 172.21.15.150:0/316310791 >> 172.21.15.150:0/316310791 conn(0x7fbeec0fd160 msgr2=0x7fbeec0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:34.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 -- 172.21.15.150:0/316310791 shutdown_connections 2024-04-26T21:10:34.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 -- 172.21.15.150:0/316310791 wait complete. 2024-04-26T21:10:34.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.826+0000 7fbef0d74700 1 Processor -- start 2024-04-26T21:10:34.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbef0d74700 1 -- start start 2024-04-26T21:10:34.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbef0d74700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 0x7fbeec105b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbef0d74700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeec102dc0 0x7fbeec1041a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbef0d74700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec103760 0x7fbeec1046e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbef0d74700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10dec0 con 0x7fbeec103760 2024-04-26T21:10:34.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbef0d74700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10dd40 con 0x7fbeec101bc0 2024-04-26T21:10:34.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbef0d74700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10e040 con 0x7fbeec102dc0 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeea59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 0x7fbeec105b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeea59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 0x7fbeec105b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34594/0 (socket says 172.21.15.150:34594) 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeea59c700 1 -- 172.21.15.150:0/793053274 learned_addr learned my addr 172.21.15.150:0/793053274 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbee9d9b700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeec102dc0 0x7fbeec1041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeead9d700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec103760 0x7fbeec1046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeea59c700 1 -- 172.21.15.150:0/793053274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeec102dc0 msgr2=0x7fbeec1041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeea59c700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeec102dc0 0x7fbeec1041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeea59c700 1 -- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec103760 msgr2=0x7fbeec1046e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeea59c700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec103760 0x7fbeec1046e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeea59c700 1 -- 172.21.15.150:0/793053274 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec104fb0 con 0x7fbeec101bc0 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbee9d9b700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeec102dc0 0x7fbeec1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeead9d700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec103760 0x7fbeec1046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbeea59c700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 0x7fbeec105b00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009f50 tx=0x7fbedc009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbedb7fe700 1 -- 172.21.15.150:0/793053274 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc027070 con 0x7fbeec101bc0 2024-04-26T21:10:34.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbedb7fe700 1 -- 172.21.15.150:0/793053274 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbedc022470 con 0x7fbeec101bc0 2024-04-26T21:10:34.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.827+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec1a3760 con 0x7fbeec101bc0 2024-04-26T21:10:34.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.828+0000 7fbedb7fe700 1 -- 172.21.15.150:0/793053274 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00f6c0 con 0x7fbeec101bc0 2024-04-26T21:10:34.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.828+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec1a3c70 con 0x7fbeec101bc0 2024-04-26T21:10:34.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.828+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec06a110 con 0x7fbeec101bc0 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.829+0000 7fbedb7fe700 1 -- 172.21.15.150:0/793053274 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fbedc02a070 con 0x7fbeec101bc0 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.830+0000 7fbedb7fe700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbed40a3bb0 0x7fbed40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.830+0000 7fbedb7fe700 1 -- 172.21.15.150:0/793053274 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fbedc0cf860 con 0x7fbeec101bc0 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.830+0000 7fbee9d9b700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbed40a3bb0 0x7fbed40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.831+0000 7fbee9d9b700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbed40a3bb0 0x7fbed40a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbee0005d90 tx=0x7fbee0005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:34.835+0000 7fbedb7fe700 1 -- 172.21.15.150:0/793053274 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbedc05f060 con 0x7fbeec101bc0 2024-04-26T21:10:35.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:35.043+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbeec1a3ed0 con 0x7fbed40a3bb0 2024-04-26T21:10:35.045 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:34 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi129 2024-04-26T21:10:35.045 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:34 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi129 to 17088M 2024-04-26T21:10:35.045 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:35.045 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:35.045 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:35.143 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:34 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi129 2024-04-26T21:10:35.143 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:34 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi129 to 17088M 2024-04-26T21:10:35.143 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:35.143 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:35.143 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:35.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:34 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi129 2024-04-26T21:10:35.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:34 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi129 to 17088M 2024-04-26T21:10:35.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:35.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:35.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:35.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:35 smithi150 ceph-mon[32290]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:10:35.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:35 smithi150 ceph-mon[32290]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:10:35.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:10:35.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:10:35.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:35.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:35 smithi150 ceph-mon[32290]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:36.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:35 smithi129 ceph-mon[29237]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:10:36.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:35 smithi129 ceph-mon[29237]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:10:36.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:10:36.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:10:36.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:36.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:35 smithi129 ceph-mon[29237]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:36.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:35 smithi171 ceph-mon[33478]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:10:36.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:35 smithi171 ceph-mon[33478]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:10:36.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:10:36.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:10:36.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:36.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:35 smithi171 ceph-mon[33478]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:38 smithi150 ceph-mon[32290]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:38 smithi150 ceph-mon[32290]: from='client.? 172.21.15.150:0/951271449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8567f6e-0353-446c-8272-3c2d98c95515"}]: dispatch 2024-04-26T21:10:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:38 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8567f6e-0353-446c-8272-3c2d98c95515"}]: dispatch 2024-04-26T21:10:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:38 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8567f6e-0353-446c-8272-3c2d98c95515"}]': finished 2024-04-26T21:10:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:38 smithi150 ceph-mon[32290]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:10:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:38 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:38 smithi129 ceph-mon[29237]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:38 smithi129 ceph-mon[29237]: from='client.? 172.21.15.150:0/951271449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8567f6e-0353-446c-8272-3c2d98c95515"}]: dispatch 2024-04-26T21:10:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:38 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8567f6e-0353-446c-8272-3c2d98c95515"}]: dispatch 2024-04-26T21:10:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:38 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8567f6e-0353-446c-8272-3c2d98c95515"}]': finished 2024-04-26T21:10:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:38 smithi129 ceph-mon[29237]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:10:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:38 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:38 smithi171 ceph-mon[33478]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:38 smithi171 ceph-mon[33478]: from='client.? 172.21.15.150:0/951271449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8567f6e-0353-446c-8272-3c2d98c95515"}]: dispatch 2024-04-26T21:10:38.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:38 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8567f6e-0353-446c-8272-3c2d98c95515"}]: dispatch 2024-04-26T21:10:38.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:38 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8567f6e-0353-446c-8272-3c2d98c95515"}]': finished 2024-04-26T21:10:38.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:38 smithi171 ceph-mon[33478]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:10:38.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:38 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:39 smithi150 ceph-mon[32290]: from='client.? 172.21.15.150:0/1909045754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:10:39.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:39 smithi129 ceph-mon[29237]: from='client.? 172.21.15.150:0/1909045754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:10:39.744 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:39 smithi171 ceph-mon[33478]: from='client.? 172.21.15.150:0/1909045754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:10:40.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:40 smithi150 ceph-mon[32290]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:40.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:40 smithi129 ceph-mon[29237]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:40 smithi171 ceph-mon[33478]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:42.526 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:42 smithi150 ceph-mon[32290]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:42 smithi129 ceph-mon[29237]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:42 smithi171 ceph-mon[33478]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:44 smithi150 ceph-mon[32290]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:44.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:44 smithi129 ceph-mon[29237]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:44 smithi171 ceph-mon[33478]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:45.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:10:45.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:45.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:10:45.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:45.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:10:45.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:46.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:46 smithi150 ceph-mon[32290]: Deploying daemon osd.1 on smithi150 2024-04-26T21:10:46.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:46 smithi150 ceph-mon[32290]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:46.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:46 smithi129 ceph-mon[29237]: Deploying daemon osd.1 on smithi150 2024-04-26T21:10:46.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:46 smithi129 ceph-mon[29237]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:46.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:46 smithi171 ceph-mon[33478]: Deploying daemon osd.1 on smithi150 2024-04-26T21:10:46.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:46 smithi171 ceph-mon[33478]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:48.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:48 smithi150 ceph-mon[32290]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:48.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:48 smithi129 ceph-mon[29237]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:48.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:48 smithi171 ceph-mon[33478]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:50.602 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:50 smithi150 ceph-mon[32290]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:50.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:50.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:50.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:50 smithi129 ceph-mon[29237]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:50 smithi171 ceph-mon[33478]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:50.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:50.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:52.609 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:52 smithi150 ceph-mon[32290]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:52 smithi129 ceph-mon[29237]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:52.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:52 smithi171 ceph-mon[33478]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:53.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.207+0000 7fbedb7fe700 1 -- 172.21.15.150:0/793053274 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbeec1a3ed0 con 0x7fbed40a3bb0 2024-04-26T21:10:53.208 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 1 on host 'smithi150' 2024-04-26T21:10:53.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbed40a3bb0 msgr2=0x7fbed40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbed40a3bb0 0x7fbed40a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbee0005d90 tx=0x7fbee0005d20 comp rx=0 tx=0).stop 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 msgr2=0x7fbeec105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 0x7fbeec105b00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009f50 tx=0x7fbedc009510 comp rx=0 tx=0).stop 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 shutdown_connections 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbed40a3bb0 0x7fbed40a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec103760 0x7fbeec1046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeec101bc0 0x7fbeec105b00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 --2- 172.21.15.150:0/793053274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeec102dc0 0x7fbeec1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 >> 172.21.15.150:0/793053274 conn(0x7fbeec0fd160 msgr2=0x7fbeec0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 shutdown_connections 2024-04-26T21:10:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:10:53.209+0000 7fbef0d74700 1 -- 172.21.15.150:0/793053274 wait complete. 2024-04-26T21:10:53.861 DEBUG:teuthology.orchestra.run.smithi150:osd.1> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T21:10:53.863 INFO:tasks.cephadm:Deploying osd.2 on smithi171 with /dev/vg_nvme/lv_4... 2024-04-26T21:10:53.864 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:10:54.111 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:10:54.470 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:54 smithi171 ceph-mon[33478]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:54.470 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.470 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.470 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.470 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.470 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:54 smithi171 ceph-mon[33478]: from='osd.1 [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:10:54.470 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:54 smithi171 ceph-mon[33478]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:10:54.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:54 smithi129 ceph-mon[29237]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:54.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:54 smithi129 ceph-mon[29237]: from='osd.1 [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:10:54.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:54 smithi129 ceph-mon[29237]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:10:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:54 smithi150 ceph-mon[32290]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:54 smithi150 ceph-mon[32290]: from='osd.1 [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:10:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:54 smithi150 ceph-mon[32290]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:10:55.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:55 smithi171 ceph-mon[33478]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:10:55.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:55 smithi171 ceph-mon[33478]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:10:55.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:55.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:55 smithi171 ceph-mon[33478]: from='osd.1 [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:10:55.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:55 smithi171 ceph-mon[33478]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:10:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:55 smithi150 ceph-mon[32290]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:10:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:55 smithi150 ceph-mon[32290]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:10:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:55 smithi150 ceph-mon[32290]: from='osd.1 [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:10:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:55 smithi150 ceph-mon[32290]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:10:55.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:55 smithi129 ceph-mon[29237]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:10:55.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:55 smithi129 ceph-mon[29237]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:10:55.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:55.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:55 smithi129 ceph-mon[29237]: from='osd.1 [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:10:55.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:55 smithi129 ceph-mon[29237]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:10:56.031 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:10:55 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1[36880]: 2024-04-26T21:10:55.732+0000 7fe3bf5f7700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T21:10:56.031 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:10:55 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1[36880]: 2024-04-26T21:10:55.735+0000 7fe3b73e2700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:10:56.141 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:10:56.141 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:10:56.141 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-26T21:10:56.141 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-26T21:10:56.141 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0179275 s, 585 MB/s 2024-04-26T21:10:56.141 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-26T21:10:56.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:56 smithi171 ceph-mon[33478]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:56.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:56 smithi171 ceph-mon[33478]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:10:56.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:56 smithi171 ceph-mon[33478]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:10:56.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:56.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:56.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:56 smithi171 ceph-mon[33478]: from='osd.1 ' entity='osd.1' 2024-04-26T21:10:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:56 smithi150 ceph-mon[32290]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:56 smithi150 ceph-mon[32290]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:10:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:56 smithi150 ceph-mon[32290]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:10:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:56 smithi150 ceph-mon[32290]: from='osd.1 ' entity='osd.1' 2024-04-26T21:10:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:56 smithi129 ceph-mon[29237]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:56 smithi129 ceph-mon[29237]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:10:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:56 smithi129 ceph-mon[29237]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:10:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:56 smithi129 ceph-mon[29237]: from='osd.1 ' entity='osd.1' 2024-04-26T21:10:56.762 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2024-04-26T21:10:57.008 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:10:57.400 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:10:57.400 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:10:57.400 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: osd.1 [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] boot 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:57.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:10:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:10:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:10:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: osd.1 [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] boot 2024-04-26T21:10:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:10:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:57.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:10:57.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:10:57.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:57.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:10:57.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: osd.1 [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] boot 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:10:57.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:10:58.418 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:58 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi150 2024-04-26T21:10:58.418 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:58 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-26T21:10:58.418 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:58 smithi171 ceph-mon[33478]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:58.419 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:58 smithi171 ceph-mon[33478]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:10:58.450 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.446+0000 7f0955fab700 1 -- 172.21.15.171:0/3514443400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950105020 msgr2=0x7f095010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:58.450 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.446+0000 7f0955fab700 1 --2- 172.21.15.171:0/3514443400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950105020 0x7f095010b8d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f094800b0a0 tx=0x7f094800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.450 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.446+0000 7f0955fab700 1 -- 172.21.15.171:0/3514443400 shutdown_connections 2024-04-26T21:10:58.450 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.446+0000 7f0955fab700 1 --2- 172.21.15.171:0/3514443400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950105020 0x7f095010b8d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.450 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.446+0000 7f0955fab700 1 --2- 172.21.15.171:0/3514443400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0950103480 0x7f09501038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.446+0000 7f0955fab700 1 --2- 172.21.15.171:0/3514443400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950104680 0x7f0950104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.446+0000 7f0955fab700 1 -- 172.21.15.171:0/3514443400 >> 172.21.15.171:0/3514443400 conn(0x7f09500fea20 msgr2=0x7f0950100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 -- 172.21.15.171:0/3514443400 shutdown_connections 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 -- 172.21.15.171:0/3514443400 wait complete. 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 Processor -- start 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 -- start start 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950103480 0x7f09501994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950104680 0x7f09501999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0950105020 0x7f09501a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f095010ef00 con 0x7f0950105020 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f095010ed80 con 0x7f0950104680 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.447+0000 7f0955fab700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f095010f080 con 0x7f0950103480 2024-04-26T21:10:58.451 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950103480 0x7f09501994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950103480 0x7f09501994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:37730/0 (socket says 172.21.15.171:37730) 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094f7fe700 1 -- 172.21.15.171:0/2965950441 learned_addr learned my addr 172.21.15.171:0/2965950441 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094ffff700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0950105020 0x7f09501a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094effd700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950104680 0x7f09501999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094f7fe700 1 -- 172.21.15.171:0/2965950441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950104680 msgr2=0x7f09501999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094f7fe700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950104680 0x7f09501999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094f7fe700 1 -- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0950105020 msgr2=0x7f09501a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094f7fe700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0950105020 0x7f09501a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094f7fe700 1 -- 172.21.15.171:0/2965950441 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09501a1110 con 0x7f0950103480 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094f7fe700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950103480 0x7f09501994b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f094400d9b0 tx=0x7f094400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094cff9700 1 -- 172.21.15.171:0/2965950441 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944017030 con 0x7f0950103480 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09501a1400 con 0x7f0950103480 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094cff9700 1 -- 172.21.15.171:0/2965950441 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09440044d0 con 0x7f0950103480 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f094cff9700 1 -- 172.21.15.171:0/2965950441 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440039e0 con 0x7f0950103480 2024-04-26T21:10:58.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.448+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09501a1990 con 0x7f0950103480 2024-04-26T21:10:58.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.449+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f095006a110 con 0x7f0950103480 2024-04-26T21:10:58.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.450+0000 7f094cff9700 1 -- 172.21.15.171:0/2965950441 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f094401a070 con 0x7f0950103480 2024-04-26T21:10:58.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.451+0000 7f094cff9700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f09380a3bb0 0x7f09380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:58.454 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.451+0000 7f094cff9700 1 -- 172.21.15.171:0/2965950441 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f09440ce4c0 con 0x7f0950103480 2024-04-26T21:10:58.454 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.451+0000 7f094effd700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f09380a3bb0 0x7f09380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:58.454 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.452+0000 7f094effd700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f09380a3bb0 0x7f09380a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0940005ea0 tx=0x7f0940005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:58.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.456+0000 7f094cff9700 1 -- 172.21.15.171:0/2965950441 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f094401ae40 con 0x7f0950103480 2024-04-26T21:10:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:58 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi150 2024-04-26T21:10:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:58 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-26T21:10:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:58 smithi150 ceph-mon[32290]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:58 smithi150 ceph-mon[32290]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:10:58.661 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:10:58.660+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0950063860 con 0x7f09380a3bb0 2024-04-26T21:10:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:58 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi150 2024-04-26T21:10:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:58 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-26T21:10:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:58 smithi129 ceph-mon[29237]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:10:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:58 smithi129 ceph-mon[29237]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:10:59.556 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:59 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:10:59.556 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:59 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:10:59.556 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:10:59 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:59.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:59 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:10:59.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:59 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:10:59.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:10:59 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:10:59.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:59 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:10:59.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:59 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:10:59.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:10:59 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:00.442 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:00 smithi171 ceph-mon[33478]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:00.442 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:00 smithi171 ceph-mon[33478]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:00.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:00 smithi150 ceph-mon[32290]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:00 smithi150 ceph-mon[32290]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:00 smithi129 ceph-mon[29237]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:00 smithi129 ceph-mon[29237]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:01.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:01 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/3496282680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "984f85b5-efab-46bb-99c0-e94a53cb512b"}]: dispatch 2024-04-26T21:11:01.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:01 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "984f85b5-efab-46bb-99c0-e94a53cb512b"}]: dispatch 2024-04-26T21:11:01.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:01 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "984f85b5-efab-46bb-99c0-e94a53cb512b"}]': finished 2024-04-26T21:11:01.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:01 smithi129 ceph-mon[29237]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:11:01.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:01.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:01 smithi150 ceph-mon[32290]: from='client.? 172.21.15.171:0/3496282680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "984f85b5-efab-46bb-99c0-e94a53cb512b"}]: dispatch 2024-04-26T21:11:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:01 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "984f85b5-efab-46bb-99c0-e94a53cb512b"}]: dispatch 2024-04-26T21:11:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:01 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "984f85b5-efab-46bb-99c0-e94a53cb512b"}]': finished 2024-04-26T21:11:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:01 smithi150 ceph-mon[32290]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:11:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:01.648 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:01 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/3496282680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "984f85b5-efab-46bb-99c0-e94a53cb512b"}]: dispatch 2024-04-26T21:11:01.648 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:01 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "984f85b5-efab-46bb-99c0-e94a53cb512b"}]: dispatch 2024-04-26T21:11:01.648 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:01 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "984f85b5-efab-46bb-99c0-e94a53cb512b"}]': finished 2024-04-26T21:11:01.648 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:01 smithi171 ceph-mon[33478]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:11:01.649 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:02 smithi150 ceph-mon[32290]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:02 smithi150 ceph-mon[32290]: from='client.? 172.21.15.171:0/3327340221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:11:02.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:02 smithi129 ceph-mon[29237]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:02.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:02 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/3327340221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:11:02.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:02 smithi171 ceph-mon[33478]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:02.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:02 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/3327340221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:11:03.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:03 smithi150 ceph-mon[32290]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:03.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:03 smithi129 ceph-mon[29237]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:03.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:03 smithi171 ceph-mon[33478]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:06.466 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:06 smithi171 ceph-mon[33478]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:06 smithi150 ceph-mon[32290]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:06.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:06 smithi129 ceph-mon[29237]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:08.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:08 smithi171 ceph-mon[33478]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:08 smithi150 ceph-mon[32290]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:08.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:08 smithi129 ceph-mon[29237]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:09.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:09 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:11:09.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:09 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:09.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:09 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:11:09.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:09 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:09.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:09 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:11:09.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:09 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:10.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:10 smithi150 ceph-mon[32290]: Deploying daemon osd.2 on smithi171 2024-04-26T21:11:10.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:10 smithi150 ceph-mon[32290]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:10 smithi129 ceph-mon[29237]: Deploying daemon osd.2 on smithi171 2024-04-26T21:11:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:10 smithi129 ceph-mon[29237]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:10 smithi171 ceph-mon[33478]: Deploying daemon osd.2 on smithi171 2024-04-26T21:11:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:10 smithi171 ceph-mon[33478]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:12.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:12 smithi171 ceph-mon[33478]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:12 smithi150 ceph-mon[32290]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:12 smithi129 ceph-mon[29237]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:14.587 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:14 smithi171 ceph-mon[33478]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:14.587 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:14 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:14.587 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:14 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:14.587 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:14 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:14 smithi150 ceph-mon[32290]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:14 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:14 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:14 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:14 smithi129 ceph-mon[29237]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:14 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:14 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:14 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:16 smithi150 ceph-mon[32290]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:16 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:16 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:16 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:16 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:16 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:16.656 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:16 smithi171 ceph-mon[33478]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:16.657 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:16 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:16.657 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:16 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:16.657 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:16 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:16.657 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:16 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:16.657 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:16 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:16 smithi129 ceph-mon[29237]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:16 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:16 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:16 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:16 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:16 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:17.027 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 2 on host 'smithi171' 2024-04-26T21:11:17.027 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.026+0000 7f094cff9700 1 -- 172.21.15.171:0/2965950441 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0950063860 con 0x7f09380a3bb0 2024-04-26T21:11:17.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f09380a3bb0 msgr2=0x7f09380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f09380a3bb0 0x7f09380a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0940005ea0 tx=0x7f0940005e30 comp rx=0 tx=0).stop 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950103480 msgr2=0x7f09501994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950103480 0x7f09501994b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f094400d9b0 tx=0x7f094400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 shutdown_connections 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f09380a3bb0 0x7f09380a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0950105020 0x7f09501a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0950103480 0x7f09501994b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 --2- 172.21.15.171:0/2965950441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950104680 0x7f09501999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.028+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 >> 172.21.15.171:0/2965950441 conn(0x7f09500fea20 msgr2=0x7f0950107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.029+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 shutdown_connections 2024-04-26T21:11:17.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:11:17.029+0000 7f0955fab700 1 -- 172.21.15.171:0/2965950441 wait complete. 2024-04-26T21:11:17.603 DEBUG:teuthology.orchestra.run.smithi171:osd.2> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T21:11:17.606 INFO:tasks.cephadm:Deploying osd.3 on smithi129 with /dev/vg_nvme/lv_3... 2024-04-26T21:11:17.606 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:11:17.852 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:18.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:18.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:18.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:18.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:18 smithi171 ceph-mon[33478]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:18.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:18 smithi171 ceph-mon[33478]: from='osd.2 [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:11:18.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:18 smithi171 ceph-mon[33478]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:11:18.277 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:18.277 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:18.277 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:18.277 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:18 smithi129 ceph-mon[29237]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:18.278 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:18 smithi129 ceph-mon[29237]: from='osd.2 [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:11:18.278 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:18 smithi129 ceph-mon[29237]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:11:18.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:18.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:18.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:18.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:18 smithi150 ceph-mon[32290]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:18.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:18 smithi150 ceph-mon[32290]: from='osd.2 [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:11:18.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:18 smithi150 ceph-mon[32290]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:11:19.273 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:19 smithi129 ceph-mon[29237]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:11:19.274 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:19 smithi129 ceph-mon[29237]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:11:19.274 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:19 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:19.274 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:19 smithi129 ceph-mon[29237]: from='osd.2 [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:11:19.274 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:19 smithi129 ceph-mon[29237]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:11:19.295 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:19 smithi171 ceph-mon[33478]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:11:19.295 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:19 smithi171 ceph-mon[33478]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:11:19.295 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:19 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:19.295 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:19 smithi171 ceph-mon[33478]: from='osd.2 [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:11:19.295 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:19 smithi171 ceph-mon[33478]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:11:19.295 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:19 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2[38635]: 2024-04-26T21:11:19.023+0000 7fc057ba0700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:11:19.295 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:19 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2[38635]: 2024-04-26T21:11:19.031+0000 7fc053993700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:19.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:19 smithi150 ceph-mon[32290]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:11:19.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:19 smithi150 ceph-mon[32290]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:11:19.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:19 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:19.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:19 smithi150 ceph-mon[32290]: from='osd.2 [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:11:19.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:19 smithi150 ceph-mon[32290]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:11:19.969 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:11:19.969 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:11:19.969 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-26T21:11:19.970 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-26T21:11:19.970 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0141347 s, 742 MB/s 2024-04-26T21:11:19.970 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-26T21:11:20.214 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:20 smithi171 ceph-mon[33478]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:11:20.214 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:20 smithi171 ceph-mon[33478]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:11:20.214 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:20 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:20.214 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:20 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:20.215 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:20 smithi171 ceph-mon[33478]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:20.219 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:20 smithi129 ceph-mon[29237]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:11:20.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:20 smithi129 ceph-mon[29237]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:11:20.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:20 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:20.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:20 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:20.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:20 smithi129 ceph-mon[29237]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:20.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:20 smithi150 ceph-mon[32290]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:11:20.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:20 smithi150 ceph-mon[32290]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:11:20.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:20 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:20.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:20 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:20.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:20 smithi150 ceph-mon[32290]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:11:20.700 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_3 2024-04-26T21:11:20.946 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:21.302 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:11:21.302 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:11:21.302 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: osd.2 [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] boot 2024-04-26T21:11:21.302 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:11:21.302 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:21.302 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.303 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.303 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:11:21.303 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.303 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:21.303 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:21.303 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:11:21.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:11:21.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: osd.2 [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] boot 2024-04-26T21:11:21.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:11:21.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:21.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:11:21.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:21.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:21.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:11:21.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:11:21.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: osd.2 [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] boot 2024-04-26T21:11:21.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:11:21.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:11:21.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:11:21.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:21.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:21.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:21.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:22.375 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:22 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi171 2024-04-26T21:11:22.376 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:22 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-26T21:11:22.376 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:22 smithi129 ceph-mon[29237]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:11:22.376 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:22 smithi129 ceph-mon[29237]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:22.376 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:11:22.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:22 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi171 2024-04-26T21:11:22.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:22 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-26T21:11:22.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:22 smithi150 ceph-mon[32290]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:11:22.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:22 smithi150 ceph-mon[32290]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:22.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:11:22.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:22 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi171 2024-04-26T21:11:22.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:22 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-26T21:11:22.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:22 smithi171 ceph-mon[33478]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:11:22.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:22 smithi171 ceph-mon[33478]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:22.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:11:22.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/2610158362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 msgr2=0x7f2ac010b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/2610158362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 0x7f2ac010b5e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800b0a0 tx=0x7f2ab800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/2610158362 shutdown_connections 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/2610158362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 0x7f2ac010b5e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/2610158362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ac01010c0 0x7f2ac01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/2610158362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ac0101a90 0x7f2ac0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/2610158362 >> 172.21.15.129:0/2610158362 conn(0x7f2ac00fcc70 msgr2=0x7f2ac00ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/2610158362 shutdown_connections 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/2610158362 wait complete. 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.602+0000 7f2ac7b4b700 1 Processor -- start 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac7b4b700 1 -- start start 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac7b4b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ac01010c0 0x7f2ac0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac7b4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ac0101a90 0x7f2ac0199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac7b4b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 0x7f2ac01a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac7b4b700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010f030 con 0x7f2ac0109140 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac7b4b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010eeb0 con 0x7f2ac0101a90 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac7b4b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010f1b0 con 0x7f2ac01010c0 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac60e8700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 0x7f2ac01a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac60e8700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 0x7f2ac01a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45870/0 (socket says 172.21.15.129:45870) 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac60e8700 1 -- 172.21.15.129:0/3637091068 learned_addr learned my addr 172.21.15.129:0/3637091068 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac50e6700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ac0101a90 0x7f2ac0199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac58e7700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ac01010c0 0x7f2ac0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac60e8700 1 -- 172.21.15.129:0/3637091068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ac01010c0 msgr2=0x7f2ac0199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac60e8700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ac01010c0 0x7f2ac0199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac60e8700 1 -- 172.21.15.129:0/3637091068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ac0101a90 msgr2=0x7f2ac0199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac60e8700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ac0101a90 0x7f2ac0199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac60e8700 1 -- 172.21.15.129:0/3637091068 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac01a1370 con 0x7f2ac0109140 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac50e6700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ac0101a90 0x7f2ac0199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac60e8700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 0x7f2ac01a0cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800cf00 tx=0x7f2ab8007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:22.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ab6ffd700 1 -- 172.21.15.129:0/3637091068 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8021070 con 0x7f2ac0109140 2024-04-26T21:11:22.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ab6ffd700 1 -- 172.21.15.129:0/3637091068 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ab801d900 con 0x7f2ac0109140 2024-04-26T21:11:22.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.603+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac01a1600 con 0x7f2ac0109140 2024-04-26T21:11:22.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.604+0000 7f2ab6ffd700 1 -- 172.21.15.129:0/3637091068 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8005460 con 0x7f2ac0109140 2024-04-26T21:11:22.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.604+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac01a1bc0 con 0x7f2ac0109140 2024-04-26T21:11:22.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.604+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac006a110 con 0x7f2ac0109140 2024-04-26T21:11:22.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.606+0000 7f2ab6ffd700 1 -- 172.21.15.129:0/3637091068 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f2ab8024070 con 0x7f2ac0109140 2024-04-26T21:11:22.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.606+0000 7f2ab6ffd700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2aac0a3bb0 0x7f2aac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.606+0000 7f2ac58e7700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2aac0a3bb0 0x7f2aac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.606+0000 7f2ab6ffd700 1 -- 172.21.15.129:0/3637091068 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f2ab800f020 con 0x7f2ac0109140 2024-04-26T21:11:22.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.607+0000 7f2ac58e7700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2aac0a3bb0 0x7f2aac0a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2abc006fd0 tx=0x7f2abc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:22.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.610+0000 7f2ab6ffd700 1 -- 172.21.15.129:0/3637091068 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ab802b020 con 0x7f2ac0109140 2024-04-26T21:11:22.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:22.819+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2ac0063860 con 0x7f2aac0a3bb0 2024-04-26T21:11:23.161 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:11:23.161 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 ceph-mon[29237]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:11:23.161 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:11:23.161 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:11:23.161 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:11:23.162 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:23.303 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42374]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:11:23.303 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42374]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.303 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42374]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.303 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42374]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.304 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:11:23.304 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 ceph-mon[33478]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:11:23.304 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:11:23.304 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:11:23.304 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:11:23.304 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:23.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:11:23.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 ceph-mon[32290]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:11:23.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:11:23.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:11:23.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:11:23.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:23.383 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40573]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:11:23.383 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40573]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.384 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40573]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.384 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40573]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.384 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40576]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:11:23.384 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.384 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40576]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.384 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40576]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.455 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48853]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:11:23.455 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.455 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48853]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.455 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48853]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.455 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48864]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:11:23.455 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48864]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.455 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48864]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.456 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48864]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.456 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48802]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:11:23.456 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48802]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.456 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48802]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.456 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48802]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.456 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48805]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:11:23.456 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48805]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.456 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48805]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.456 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:11:23 smithi129 sudo[48805]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.742 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42377]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:11:23.742 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42377]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.742 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42377]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.742 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42377]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42380]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:11:23.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42380]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42380]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42380]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42383]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:11:23.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42383]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42383]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:23 smithi171 sudo[42383]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40579]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:11:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40579]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40579]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40582]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:11:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40582]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:11:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40582]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:11:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:23 smithi150 sudo[40582]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:11:24.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:24.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:11:24.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:11:24.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:24.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:24.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:24.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:24.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:11:24.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:11:24.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:11:24.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:24 smithi129 ceph-mon[29237]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:11:24.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:24.219+0000 7f2ab6ffd700 1 -- 172.21.15.129:0/3637091068 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2ab801dab0 con 0x7f2ac0109140 2024-04-26T21:11:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:11:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:11:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:11:24.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:11:24.385 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:11:24.385 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:24 smithi150 ceph-mon[32290]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:11:24.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:11:24.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:11:24.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:24 smithi171 ceph-mon[33478]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:11:25.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:25 smithi129 ceph-mon[29237]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T21:11:25.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:25 smithi129 ceph-mon[29237]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T21:11:25.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:25 smithi150 ceph-mon[32290]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T21:11:25.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:25 smithi150 ceph-mon[32290]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T21:11:25.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:25 smithi171 ceph-mon[33478]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T21:11:25.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:25 smithi171 ceph-mon[33478]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T21:11:26.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:26 smithi150 ceph-mon[32290]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:26.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:26 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1570626674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e52e54-d5ed-417d-957b-e951ff05b920"}]: dispatch 2024-04-26T21:11:26.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:26 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1570626674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04e52e54-d5ed-417d-957b-e951ff05b920"}]': finished 2024-04-26T21:11:26.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:26 smithi150 ceph-mon[32290]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:11:26.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:26 smithi129 ceph-mon[29237]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:26 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1570626674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e52e54-d5ed-417d-957b-e951ff05b920"}]: dispatch 2024-04-26T21:11:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:26 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1570626674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04e52e54-d5ed-417d-957b-e951ff05b920"}]': finished 2024-04-26T21:11:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:26 smithi129 ceph-mon[29237]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:11:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:26.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:26 smithi171 ceph-mon[33478]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:26.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:26 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1570626674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e52e54-d5ed-417d-957b-e951ff05b920"}]: dispatch 2024-04-26T21:11:26.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:26 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1570626674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04e52e54-d5ed-417d-957b-e951ff05b920"}]': finished 2024-04-26T21:11:26.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:26 smithi171 ceph-mon[33478]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:11:26.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:27.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:27 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2743878642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:11:27.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:27 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2743878642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:11:27.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:27 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2743878642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:11:28.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:28 smithi150 ceph-mon[32290]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:28.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:28 smithi129 ceph-mon[29237]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:28 smithi171 ceph-mon[33478]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:30.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:30 smithi150 ceph-mon[32290]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:30.634 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:30 smithi129 ceph-mon[29237]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:30 smithi171 ceph-mon[33478]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:32 smithi150 ceph-mon[32290]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:32.712 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:32 smithi129 ceph-mon[29237]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:32.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:32 smithi171 ceph-mon[33478]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:33.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:11:33.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:11:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:33.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:11:33.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:34 smithi150 ceph-mon[32290]: Deploying daemon osd.3 on smithi129 2024-04-26T21:11:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:34 smithi150 ceph-mon[32290]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:34 smithi129 ceph-mon[29237]: Deploying daemon osd.3 on smithi129 2024-04-26T21:11:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:34 smithi129 ceph-mon[29237]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:34.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:34 smithi171 ceph-mon[33478]: Deploying daemon osd.3 on smithi129 2024-04-26T21:11:34.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:34 smithi171 ceph-mon[33478]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:36.543 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:36 smithi171 ceph-mon[33478]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:36 smithi150 ceph-mon[32290]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:36.687 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:36 smithi129 ceph-mon[29237]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:38 smithi129 ceph-mon[29237]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:38 smithi150 ceph-mon[32290]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:38 smithi171 ceph-mon[33478]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:40.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:40 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:40.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:40 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:40.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:40 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:40.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:40 smithi150 ceph-mon[32290]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:40.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:40 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:40.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:40 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:40.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:40 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:40.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:40 smithi129 ceph-mon[29237]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:40.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:40.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:40.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:40.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:40 smithi171 ceph-mon[33478]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:42.562 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:42 smithi129 ceph-mon[29237]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:42.565 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:42.565 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:42.565 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:42.565 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:42.565 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:42.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:42 smithi150 ceph-mon[32290]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:42 smithi171 ceph-mon[33478]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:42.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:42.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:42.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:42.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:42.937 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 3 on host 'smithi129' 2024-04-26T21:11:42.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.935+0000 7f2ab6ffd700 1 -- 172.21.15.129:0/3637091068 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2ac0063860 con 0x7f2aac0a3bb0 2024-04-26T21:11:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2aac0a3bb0 msgr2=0x7f2aac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2aac0a3bb0 0x7f2aac0a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2abc006fd0 tx=0x7f2abc008040 comp rx=0 tx=0).stop 2024-04-26T21:11:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 msgr2=0x7f2ac01a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 0x7f2ac01a0cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800cf00 tx=0x7f2ab8007770 comp rx=0 tx=0).stop 2024-04-26T21:11:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 shutdown_connections 2024-04-26T21:11:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2aac0a3bb0 0x7f2aac0a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ac0109140 0x7f2ac01a0cd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ac01010c0 0x7f2ac0199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:42.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 --2- 172.21.15.129:0/3637091068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ac0101a90 0x7f2ac0199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:42.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 >> 172.21.15.129:0/3637091068 conn(0x7f2ac00fcc70 msgr2=0x7f2ac0104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:42.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 shutdown_connections 2024-04-26T21:11:42.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:11:42.938+0000 7f2ac7b4b700 1 -- 172.21.15.129:0/3637091068 wait complete. 2024-04-26T21:11:43.601 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:43 smithi129 ceph-mon[29237]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:11:43.601 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:43 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:43.601 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:43 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:43.601 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:43 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:43.627 DEBUG:teuthology.orchestra.run.smithi129:osd.3> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T21:11:43.629 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_3... 2024-04-26T21:11:43.630 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:11:43.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:43 smithi150 ceph-mon[32290]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:11:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:43 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:43 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:43 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:43 smithi171 ceph-mon[33478]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:11:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:43 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:43 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:43.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:43 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:11:43.877 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:11:44.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:44 smithi129 ceph-mon[29237]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:44.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:44 smithi129 ceph-mon[29237]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:11:44.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:44 smithi129 ceph-mon[29237]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:11:44.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:44 smithi129 ceph-mon[29237]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:11:44.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:44 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:44.471 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:11:44 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3[50984]: 2024-04-26T21:11:44.272+0000 7fc007780700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T21:11:44.471 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:11:44 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3[50984]: 2024-04-26T21:11:44.279+0000 7fc003573700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:44.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:44 smithi150 ceph-mon[32290]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:44.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:44 smithi150 ceph-mon[32290]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:11:44.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:44 smithi150 ceph-mon[32290]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:11:44.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:44 smithi150 ceph-mon[32290]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:11:44.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:44 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:44 smithi171 ceph-mon[33478]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:44 smithi171 ceph-mon[33478]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:11:44.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:44 smithi171 ceph-mon[33478]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:11:44.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:44 smithi171 ceph-mon[33478]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:11:44.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:44 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:45.519 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:45 smithi129 ceph-mon[29237]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:11:45.519 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:45 smithi129 ceph-mon[29237]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T21:11:45.519 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:45.519 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:45.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:45 smithi150 ceph-mon[32290]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:11:45.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:45 smithi150 ceph-mon[32290]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T21:11:45.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:45.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:45.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:45 smithi171 ceph-mon[33478]: from='osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:11:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:45 smithi171 ceph-mon[33478]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T21:11:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:45.872 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:11:45.872 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:11:45.872 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-26T21:11:45.872 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-26T21:11:45.872 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121353 s, 864 MB/s 2024-04-26T21:11:45.872 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-26T21:11:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:46 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:11:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:46 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:11:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:46 smithi129 ceph-mon[29237]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:46 smithi129 ceph-mon[29237]: osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] boot 2024-04-26T21:11:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:46 smithi129 ceph-mon[29237]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:11:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:46 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:46 smithi129 ceph-mon[29237]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:11:46.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:46 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:11:46.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:46 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:11:46.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:46 smithi150 ceph-mon[32290]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:46.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:46 smithi150 ceph-mon[32290]: osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] boot 2024-04-26T21:11:46.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:46 smithi150 ceph-mon[32290]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:11:46.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:46 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:46.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:46 smithi150 ceph-mon[32290]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:11:46.497 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-04-26T21:11:46.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:46 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:11:46.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:46 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:11:46.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:46 smithi171 ceph-mon[33478]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:11:46.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:46 smithi171 ceph-mon[33478]: osd.3 [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] boot 2024-04-26T21:11:46.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:46 smithi171 ceph-mon[33478]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:11:46.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:46 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:11:46.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:46 smithi171 ceph-mon[33478]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:11:46.743 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:11:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:11:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi129 2024-04-26T21:11:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:11:47.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:11:47.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi129 to 8544M 2024-04-26T21:11:47.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:47.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:47.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:47 smithi150 ceph-mon[32290]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:11:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:11:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi129 2024-04-26T21:11:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:11:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:11:47.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi129 to 8544M 2024-04-26T21:11:47.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:47.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:47.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:47 smithi129 ceph-mon[29237]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:11:47.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi129 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi129 to 8544M 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:11:47.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:47 smithi171 ceph-mon[33478]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:11:48.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.449+0000 7f4656754700 1 -- 172.21.15.150:0/2474609505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 msgr2=0x7f465010b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:48.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.449+0000 7f4656754700 1 --2- 172.21.15.150:0/2474609505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 0x7f465010b640 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f464800b0a0 tx=0x7f464800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:11:48.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.449+0000 7f4656754700 1 -- 172.21.15.150:0/2474609505 shutdown_connections 2024-04-26T21:11:48.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.449+0000 7f4656754700 1 --2- 172.21.15.150:0/2474609505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 0x7f465010b640 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:48.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.449+0000 7f4656754700 1 --2- 172.21.15.150:0/2474609505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4650103ff0 0x7f46501063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:48.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.449+0000 7f4656754700 1 --2- 172.21.15.150:0/2474609505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4650106920 0x7f4650108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:48.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.449+0000 7f4656754700 1 -- 172.21.15.150:0/2474609505 >> 172.21.15.150:0/2474609505 conn(0x7f46500fd980 msgr2=0x7f46500ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:48.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.450+0000 7f4656754700 1 -- 172.21.15.150:0/2474609505 shutdown_connections 2024-04-26T21:11:48.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.450+0000 7f4656754700 1 -- 172.21.15.150:0/2474609505 wait complete. 2024-04-26T21:11:48.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.450+0000 7f4656754700 1 Processor -- start 2024-04-26T21:11:48.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.450+0000 7f4656754700 1 -- start start 2024-04-26T21:11:48.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4656754700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4650103ff0 0x7f46501995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:48.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4656754700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4650106920 0x7f4650199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:48.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4656754700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 0x7f46501a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:48.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4656754700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f465010ef70 con 0x7f4650106920 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4656754700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f465010edf0 con 0x7f4650109250 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4656754700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f465010f0f0 con 0x7f4650103ff0 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4654cf1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 0x7f46501a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4654cf1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 0x7f46501a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59802/0 (socket says 172.21.15.150:59802) 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4654cf1700 1 -- 172.21.15.150:0/617560704 learned_addr learned my addr 172.21.15.150:0/617560704 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f464ffff700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4650103ff0 0x7f46501995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f464f7fe700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4650106920 0x7f4650199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4654cf1700 1 -- 172.21.15.150:0/617560704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4650103ff0 msgr2=0x7f46501995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4654cf1700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4650103ff0 0x7f46501995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4654cf1700 1 -- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4650106920 msgr2=0x7f4650199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4654cf1700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4650106920 0x7f4650199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4654cf1700 1 -- 172.21.15.150:0/617560704 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46501a1240 con 0x7f4650109250 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f464f7fe700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4650106920 0x7f4650199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:48.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f464ffff700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4650103ff0 0x7f46501995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:48.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f4654cf1700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 0x7f46501a0ba0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4648009fd0 tx=0x7f4648007770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:48.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.451+0000 7f464d7fa700 1 -- 172.21.15.150:0/617560704 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648021030 con 0x7f4650109250 2024-04-26T21:11:48.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.452+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46501a14d0 con 0x7f4650109250 2024-04-26T21:11:48.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.452+0000 7f464d7fa700 1 -- 172.21.15.150:0/617560704 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4648007d20 con 0x7f4650109250 2024-04-26T21:11:48.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.452+0000 7f464d7fa700 1 -- 172.21.15.150:0/617560704 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464801d470 con 0x7f4650109250 2024-04-26T21:11:48.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.452+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46501a1980 con 0x7f4650109250 2024-04-26T21:11:48.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.453+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465006a110 con 0x7f4650109250 2024-04-26T21:11:48.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.453+0000 7f464d7fa700 1 -- 172.21.15.150:0/617560704 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4648024070 con 0x7f4650109250 2024-04-26T21:11:48.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.454+0000 7f464d7fa700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46380a3cc0 0x7f46380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:48.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.454+0000 7f464d7fa700 1 -- 172.21.15.150:0/617560704 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f46480d04e0 con 0x7f4650109250 2024-04-26T21:11:48.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.455+0000 7f464ffff700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46380a3cc0 0x7f46380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:48.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.455+0000 7f464ffff700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46380a3cc0 0x7f46380a6180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4640005950 tx=0x7f4640005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:48.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.459+0000 7f464d7fa700 1 -- 172.21.15.150:0/617560704 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f464809e170 con 0x7f4650109250 2024-04-26T21:11:48.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:11:48.667+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4650063860 con 0x7f46380a3cc0 2024-04-26T21:11:48.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:48 smithi150 ceph-mon[32290]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:11:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:48 smithi129 ceph-mon[29237]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:11:48.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:48 smithi171 ceph-mon[33478]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:11:49.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:49 smithi150 ceph-mon[32290]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T21:11:49.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:49 smithi150 ceph-mon[32290]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:49.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:49 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:11:49.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:49 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:11:49.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:49 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:49.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:49 smithi150 ceph-mon[32290]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-26T21:11:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:49 smithi129 ceph-mon[29237]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T21:11:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:49 smithi129 ceph-mon[29237]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:49 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:11:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:49 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:11:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:49 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:49 smithi129 ceph-mon[29237]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-26T21:11:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:49 smithi171 ceph-mon[33478]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T21:11:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:49 smithi171 ceph-mon[33478]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:49 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:11:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:49 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:11:49.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:49 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:49.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:49 smithi171 ceph-mon[33478]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-26T21:11:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:52 smithi150 ceph-mon[32290]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-26T21:11:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:52 smithi150 ceph-mon[32290]: from='client.? 172.21.15.150:0/1242033075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc2cdc94-fc55-4068-b8a0-0119fcb9dd39"}]: dispatch 2024-04-26T21:11:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:52 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc2cdc94-fc55-4068-b8a0-0119fcb9dd39"}]: dispatch 2024-04-26T21:11:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:52 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc2cdc94-fc55-4068-b8a0-0119fcb9dd39"}]': finished 2024-04-26T21:11:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:52 smithi150 ceph-mon[32290]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:11:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:52 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:11:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:52 smithi150 ceph-mon[32290]: from='client.? 172.21.15.150:0/3516972429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:11:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:52 smithi129 ceph-mon[29237]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-26T21:11:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:52 smithi129 ceph-mon[29237]: from='client.? 172.21.15.150:0/1242033075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc2cdc94-fc55-4068-b8a0-0119fcb9dd39"}]: dispatch 2024-04-26T21:11:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:52 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc2cdc94-fc55-4068-b8a0-0119fcb9dd39"}]: dispatch 2024-04-26T21:11:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:52 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc2cdc94-fc55-4068-b8a0-0119fcb9dd39"}]': finished 2024-04-26T21:11:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:52 smithi129 ceph-mon[29237]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:11:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:52 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:11:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:52 smithi129 ceph-mon[29237]: from='client.? 172.21.15.150:0/3516972429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:11:52.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:52 smithi171 ceph-mon[33478]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-26T21:11:52.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:52 smithi171 ceph-mon[33478]: from='client.? 172.21.15.150:0/1242033075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc2cdc94-fc55-4068-b8a0-0119fcb9dd39"}]: dispatch 2024-04-26T21:11:52.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:52 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc2cdc94-fc55-4068-b8a0-0119fcb9dd39"}]: dispatch 2024-04-26T21:11:52.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:52 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc2cdc94-fc55-4068-b8a0-0119fcb9dd39"}]': finished 2024-04-26T21:11:52.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:52 smithi171 ceph-mon[33478]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:11:52.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:52 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:11:52.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:52 smithi171 ceph-mon[33478]: from='client.? 172.21.15.150:0/3516972429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:11:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:54 smithi150 ceph-mon[32290]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:11:54.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:54 smithi129 ceph-mon[29237]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:11:54.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:54 smithi171 ceph-mon[33478]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:11:56.548 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:56 smithi150 ceph-mon[32290]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:11:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:56 smithi129 ceph-mon[29237]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:11:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:56 smithi171 ceph-mon[33478]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:11:58.542 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:58 smithi150 ceph-mon[32290]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:11:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:58 smithi129 ceph-mon[29237]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:11:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:58 smithi171 ceph-mon[33478]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:11:59.494 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:59 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:11:59.494 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:11:59 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:59.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:59 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:11:59.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:11:59 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:59.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:59 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:11:59.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:11:59 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:00 smithi150 ceph-mon[32290]: Deploying daemon osd.4 on smithi150 2024-04-26T21:12:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:00 smithi150 ceph-mon[32290]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:00 smithi129 ceph-mon[29237]: Deploying daemon osd.4 on smithi150 2024-04-26T21:12:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:00 smithi129 ceph-mon[29237]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:00 smithi171 ceph-mon[33478]: Deploying daemon osd.4 on smithi150 2024-04-26T21:12:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:00 smithi171 ceph-mon[33478]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:02.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:02 smithi150 ceph-mon[32290]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:02.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:02 smithi171 ceph-mon[33478]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:02.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:02 smithi129 ceph-mon[29237]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:04 smithi150 ceph-mon[32290]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:04 smithi129 ceph-mon[29237]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:04.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:04 smithi171 ceph-mon[33478]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:04.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:04.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:06.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:06 smithi150 ceph-mon[32290]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:06.544 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:06 smithi171 ceph-mon[33478]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:06.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:06 smithi129 ceph-mon[29237]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:07.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.530+0000 7f464d7fa700 1 -- 172.21.15.150:0/617560704 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4650063860 con 0x7f46380a3cc0 2024-04-26T21:12:07.532 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-04-26T21:12:07.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.532+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46380a3cc0 msgr2=0x7f46380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:07.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.532+0000 7f4656754700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46380a3cc0 0x7f46380a6180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4640005950 tx=0x7f4640005f50 comp rx=0 tx=0).stop 2024-04-26T21:12:07.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.532+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 msgr2=0x7f46501a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:07.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.532+0000 7f4656754700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 0x7f46501a0ba0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4648009fd0 tx=0x7f4648007770 comp rx=0 tx=0).stop 2024-04-26T21:12:07.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.533+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 shutdown_connections 2024-04-26T21:12:07.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.533+0000 7f4656754700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46380a3cc0 0x7f46380a6180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:07.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.533+0000 7f4656754700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650109250 0x7f46501a0ba0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:07.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.533+0000 7f4656754700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4650103ff0 0x7f46501995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:07.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.533+0000 7f4656754700 1 --2- 172.21.15.150:0/617560704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4650106920 0x7f4650199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:07.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.533+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 >> 172.21.15.150:0/617560704 conn(0x7f46500fd980 msgr2=0x7f46501048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:07.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.533+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 shutdown_connections 2024-04-26T21:12:07.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:12:07.533+0000 7f4656754700 1 -- 172.21.15.150:0/617560704 wait complete. 2024-04-26T21:12:08.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:08.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:08.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:07 smithi150 ceph-mon[32290]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:08.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:08.146 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T21:12:08.148 INFO:tasks.cephadm:Deploying osd.5 on smithi171 with /dev/vg_nvme/lv_3... 2024-04-26T21:12:08.148 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:12:08.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:08.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:08.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:07 smithi171 ceph-mon[33478]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:08.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:08.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:08.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:08.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:07 smithi129 ceph-mon[29237]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:08.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:08.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:08.378 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:12:09.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:08 smithi150 ceph-mon[32290]: from='osd.4 [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:12:09.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:08 smithi150 ceph-mon[32290]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:12:09.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:08 smithi129 ceph-mon[29237]: from='osd.4 [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:12:09.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:08 smithi129 ceph-mon[29237]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:12:09.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:08 smithi171 ceph-mon[33478]: from='osd.4 [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:12:09.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:08 smithi171 ceph-mon[33478]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:12:09.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:09 smithi171 ceph-mon[33478]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:12:09.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:09 smithi171 ceph-mon[33478]: from='osd.4 [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:12:09.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:09 smithi171 ceph-mon[33478]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:12:09.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:09 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:09.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:09 smithi171 ceph-mon[33478]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:12:09.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:09 smithi171 ceph-mon[33478]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:09 smithi150 ceph-mon[32290]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:12:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:09 smithi150 ceph-mon[32290]: from='osd.4 [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:12:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:09 smithi150 ceph-mon[32290]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:12:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:09 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:09 smithi150 ceph-mon[32290]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:12:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:09 smithi150 ceph-mon[32290]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:10.134 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:12:09 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4[43380]: 2024-04-26T21:12:09.878+0000 7f50b7b72700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:12:10.134 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:12:09 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4[43380]: 2024-04-26T21:12:09.884+0000 7f50b3965700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:12:10.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:09 smithi129 ceph-mon[29237]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:12:10.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:09 smithi129 ceph-mon[29237]: from='osd.4 [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:12:10.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:09 smithi129 ceph-mon[29237]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:12:10.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:09 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:10.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:09 smithi129 ceph-mon[29237]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:12:10.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:09 smithi129 ceph-mon[29237]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:12:10.229 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:12:10.229 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:12:10.229 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-26T21:12:10.229 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-26T21:12:10.230 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114873 s, 913 MB/s 2024-04-26T21:12:10.230 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-26T21:12:10.870 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2024-04-26T21:12:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:10 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:12:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:10 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:12:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:10 smithi150 ceph-mon[32290]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:12:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:10 smithi150 ceph-mon[32290]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T21:12:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:11.144 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:12:11.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:10 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:12:11.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:10 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:12:11.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:10 smithi129 ceph-mon[29237]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:12:11.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:10 smithi129 ceph-mon[29237]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T21:12:11.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:11.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:11.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:10 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:12:11.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:10 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:12:11.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:10 smithi171 ceph-mon[33478]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:12:11.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:10 smithi171 ceph-mon[33478]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T21:12:11.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:11.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:12.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: osd.4 [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] boot 2024-04-26T21:12:12.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:12:12.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:12.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:12.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi150 2024-04-26T21:12:12.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:12.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:12.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-26T21:12:12.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:12.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:12.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: osd.4 [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] boot 2024-04-26T21:12:12.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi150 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:12.223 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: osd.4 [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] boot 2024-04-26T21:12:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:12:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi150 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:12.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 -- 172.21.15.171:0/1595238688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 msgr2=0x7f1de810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1595238688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 0x7f1de810b8d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1de000b0a0 tx=0x7f1de000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 -- 172.21.15.171:0/1595238688 shutdown_connections 2024-04-26T21:12:12.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1595238688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 0x7f1de810b8d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1595238688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de8103480 0x7f1de81038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1595238688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de8104680 0x7f1de8104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 -- 172.21.15.171:0/1595238688 >> 172.21.15.171:0/1595238688 conn(0x7f1de80fea20 msgr2=0x7f1de8100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 -- 172.21.15.171:0/1595238688 shutdown_connections 2024-04-26T21:12:12.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 -- 172.21.15.171:0/1595238688 wait complete. 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.644+0000 7f1deeb03700 1 Processor -- start 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1deeb03700 1 -- start start 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1deeb03700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de8103480 0x7f1de8199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1deeb03700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de8104680 0x7f1de8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1deeb03700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 0x7f1de81a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1deeb03700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1de810ef20 con 0x7f1de8103480 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1deeb03700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1de810eda0 con 0x7f1de8104680 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1deeb03700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1de810f0a0 con 0x7f1de8105020 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1ded0a0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 0x7f1de81a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1ded0a0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 0x7f1de81a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:54522/0 (socket says 172.21.15.171:54522) 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1ded0a0700 1 -- 172.21.15.171:0/1197020535 learned_addr learned my addr 172.21.15.171:0/1197020535 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1dec89f700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de8103480 0x7f1de8199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1de7fff700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de8104680 0x7f1de8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1ded0a0700 1 -- 172.21.15.171:0/1197020535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de8104680 msgr2=0x7f1de8199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1ded0a0700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de8104680 0x7f1de8199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1ded0a0700 1 -- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de8103480 msgr2=0x7f1de8199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1ded0a0700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de8103480 0x7f1de8199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1ded0a0700 1 -- 172.21.15.171:0/1197020535 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de8075650 con 0x7f1de8105020 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1ded0a0700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 0x7f1de81a0b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1de0004e40 tx=0x7f1de0004e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1de5ffb700 1 -- 172.21.15.171:0/1197020535 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0021070 con 0x7f1de8105020 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1de5ffb700 1 -- 172.21.15.171:0/1197020535 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1de0007c80 con 0x7f1de8105020 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1de5ffb700 1 -- 172.21.15.171:0/1197020535 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0004850 con 0x7f1de8105020 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.645+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de80758e0 con 0x7f1de8105020 2024-04-26T21:12:12.649 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.646+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de8075e20 con 0x7f1de8105020 2024-04-26T21:12:12.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.647+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de806a110 con 0x7f1de8105020 2024-04-26T21:12:12.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.648+0000 7f1de5ffb700 1 -- 172.21.15.171:0/1197020535 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1de0004020 con 0x7f1de8105020 2024-04-26T21:12:12.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.648+0000 7f1de5ffb700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1dd00a3cc0 0x7f1dd00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.648+0000 7f1de5ffb700 1 -- 172.21.15.171:0/1197020535 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f1de00d04a0 con 0x7f1de8105020 2024-04-26T21:12:12.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.648+0000 7f1dec89f700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1dd00a3cc0 0x7f1dd00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.651 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.649+0000 7f1dec89f700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1dd00a3cc0 0x7f1dd00a6180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1de81044e0 tx=0x7f1dd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.654 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.653+0000 7f1de5ffb700 1 -- 172.21.15.171:0/1197020535 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1de009df80 con 0x7f1de8105020 2024-04-26T21:12:12.862 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:12.861+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1de810a310 con 0x7f1dd00a3cc0 2024-04-26T21:12:13.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:12 smithi129 ceph-mon[29237]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T21:12:13.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:12:13.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:12:13.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:13.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:12 smithi171 ceph-mon[33478]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T21:12:13.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:12 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:12:13.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:12 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:12:13.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:12 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:13.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:12 smithi150 ceph-mon[32290]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T21:12:13.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:12 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:12:13.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:12 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:12:13.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:12 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:14.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:13 smithi171 ceph-mon[33478]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:14.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:13 smithi171 ceph-mon[33478]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:14.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:13 smithi129 ceph-mon[29237]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:14.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:13 smithi129 ceph-mon[29237]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:14.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:13 smithi150 ceph-mon[32290]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:14.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:13 smithi150 ceph-mon[32290]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:16 smithi150 ceph-mon[32290]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:16 smithi150 ceph-mon[32290]: from='client.? 172.21.15.171:0/3706214623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09db0a71-d9ab-4959-9fb8-cee38f030d2d"}]: dispatch 2024-04-26T21:12:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:16 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09db0a71-d9ab-4959-9fb8-cee38f030d2d"}]: dispatch 2024-04-26T21:12:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:16 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09db0a71-d9ab-4959-9fb8-cee38f030d2d"}]': finished 2024-04-26T21:12:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:16 smithi150 ceph-mon[32290]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:12:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:16 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:16.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:16 smithi150 ceph-mon[32290]: from='client.? 172.21.15.171:0/2045577389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:12:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:16 smithi129 ceph-mon[29237]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:16 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/3706214623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09db0a71-d9ab-4959-9fb8-cee38f030d2d"}]: dispatch 2024-04-26T21:12:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:16 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09db0a71-d9ab-4959-9fb8-cee38f030d2d"}]: dispatch 2024-04-26T21:12:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:16 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09db0a71-d9ab-4959-9fb8-cee38f030d2d"}]': finished 2024-04-26T21:12:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:16 smithi129 ceph-mon[29237]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:12:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:16 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:16 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/2045577389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:12:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:16 smithi171 ceph-mon[33478]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:16 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/3706214623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09db0a71-d9ab-4959-9fb8-cee38f030d2d"}]: dispatch 2024-04-26T21:12:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:16 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09db0a71-d9ab-4959-9fb8-cee38f030d2d"}]: dispatch 2024-04-26T21:12:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:16 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09db0a71-d9ab-4959-9fb8-cee38f030d2d"}]': finished 2024-04-26T21:12:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:16 smithi171 ceph-mon[33478]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:12:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:16 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:16 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/2045577389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:12:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:18 smithi150 ceph-mon[32290]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:18.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:18 smithi129 ceph-mon[29237]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:18.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:18 smithi171 ceph-mon[33478]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:20.593 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:20 smithi171 ceph-mon[33478]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:20 smithi150 ceph-mon[32290]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:20.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:20 smithi129 ceph-mon[29237]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:22.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:22 smithi171 ceph-mon[33478]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:22 smithi150 ceph-mon[32290]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:22.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:22 smithi129 ceph-mon[29237]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:23.596 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:12:23.596 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:12:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:12:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:24.374 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:24 smithi171 ceph-mon[33478]: Deploying daemon osd.5 on smithi171 2024-04-26T21:12:24.375 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:24 smithi171 ceph-mon[33478]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:24.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:24 smithi150 ceph-mon[32290]: Deploying daemon osd.5 on smithi171 2024-04-26T21:12:24.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:24 smithi150 ceph-mon[32290]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:24 smithi129 ceph-mon[29237]: Deploying daemon osd.5 on smithi171 2024-04-26T21:12:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:24 smithi129 ceph-mon[29237]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:26.536 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:26 smithi171 ceph-mon[33478]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:26.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:26 smithi150 ceph-mon[32290]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:26 smithi129 ceph-mon[29237]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:28.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:28 smithi150 ceph-mon[32290]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:28.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:28 smithi129 ceph-mon[29237]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:28 smithi171 ceph-mon[33478]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:29.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:29 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:29.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:29 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:29.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:29 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:29.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:29 smithi129 ceph-mon[29237]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:29.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:29 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:29.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:29 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:29.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:29 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:29.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:29 smithi171 ceph-mon[33478]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:29.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:29 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:29.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:29 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:29.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:29 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:29.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:29 smithi150 ceph-mon[32290]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:32.330 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.328+0000 7f1de5ffb700 1 -- 172.21.15.171:0/1197020535 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1de810a310 con 0x7f1dd00a3cc0 2024-04-26T21:12:32.331 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 5 on host 'smithi171' 2024-04-26T21:12:32.333 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1dd00a3cc0 msgr2=0x7f1dd00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.333 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1dd00a3cc0 0x7f1dd00a6180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1de81044e0 tx=0x7f1dd800b410 comp rx=0 tx=0).stop 2024-04-26T21:12:32.333 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 msgr2=0x7f1de81a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.333 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 0x7f1de81a0b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1de0004e40 tx=0x7f1de0004e70 comp rx=0 tx=0).stop 2024-04-26T21:12:32.333 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 shutdown_connections 2024-04-26T21:12:32.333 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1dd00a3cc0 0x7f1dd00a6180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.333 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de8105020 0x7f1de81a0b50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.333 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de8103480 0x7f1de8199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.333 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 --2- 172.21.15.171:0/1197020535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de8104680 0x7f1de8199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.334 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 >> 172.21.15.171:0/1197020535 conn(0x7f1de80fea20 msgr2=0x7f1de8107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:32.334 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 shutdown_connections 2024-04-26T21:12:32.334 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:12:32.331+0000 7f1deeb03700 1 -- 172.21.15.171:0/1197020535 wait complete. 2024-04-26T21:12:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:32 smithi150 ceph-mon[32290]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:32 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:32 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:32 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:32 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:32 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:32.645 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:32 smithi171 ceph-mon[33478]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:32.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:32 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:32.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:32 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:32.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:32 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:32.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:32 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:32.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:32 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:32 smithi129 ceph-mon[29237]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:32 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:32 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:32 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:32 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:32 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:32.978 DEBUG:teuthology.orchestra.run.smithi171:osd.5> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T21:12:32.980 INFO:tasks.cephadm:Deploying osd.6 on smithi129 with /dev/vg_nvme/lv_2... 2024-04-26T21:12:32.981 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:12:33.225 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:12:33.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:33.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:33.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:33.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:33 smithi171 ceph-mon[33478]: from='osd.5 [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:12:33.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:33 smithi171 ceph-mon[33478]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:12:33.608 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:33.608 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:33.608 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:33.608 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:33 smithi129 ceph-mon[29237]: from='osd.5 [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:12:33.608 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:33 smithi129 ceph-mon[29237]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:12:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:33 smithi150 ceph-mon[32290]: from='osd.5 [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:12:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:33 smithi150 ceph-mon[32290]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:12:34.322 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:34 smithi171 ceph-mon[33478]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:34.322 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:34 smithi171 ceph-mon[33478]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:12:34.322 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:34 smithi171 ceph-mon[33478]: from='osd.5 [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:12:34.322 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:34 smithi171 ceph-mon[33478]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:12:34.322 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:34.322 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:34 smithi171 ceph-mon[33478]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:12:34.323 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:12:34 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:12:34.300+0000 7f214fd25700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:12:34.323 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:12:34 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:12:34.307+0000 7f214bb18700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:12:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:34 smithi150 ceph-mon[32290]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:34 smithi150 ceph-mon[32290]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:12:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:34 smithi150 ceph-mon[32290]: from='osd.5 [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:12:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:34 smithi150 ceph-mon[32290]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:12:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:34.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:34 smithi150 ceph-mon[32290]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:12:34.635 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:34 smithi129 ceph-mon[29237]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:34.635 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:34 smithi129 ceph-mon[29237]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:12:34.635 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:34 smithi129 ceph-mon[29237]: from='osd.5 [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:12:34.635 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:34 smithi129 ceph-mon[29237]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:12:34.635 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:34.636 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:34 smithi129 ceph-mon[29237]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:12:35.374 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:12:35.374 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:12:35.374 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-26T21:12:35.374 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-26T21:12:35.374 INFO:teuthology.orchestra.run.smithi129.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122115 s, 859 MB/s 2024-04-26T21:12:35.374 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 2024-04-26T21:12:35.374 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-26T21:12:35.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:35 smithi150 ceph-mon[32290]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:12:35.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:35 smithi150 ceph-mon[32290]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:12:35.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:35.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:35.672 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:35 smithi129 ceph-mon[29237]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:12:35.672 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:35 smithi129 ceph-mon[29237]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:12:35.672 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:35.672 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:35.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:35 smithi171 ceph-mon[33478]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:12:35.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:35 smithi171 ceph-mon[33478]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:12:35.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:35.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:36.115 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_2 2024-04-26T21:12:36.378 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:12:36.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:12:36.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:12:36.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:36.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:36.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: osd.5 [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] boot 2024-04-26T21:12:36.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:12:36.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:36.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:12:36.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:36.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:36.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:36.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:36.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: osd.5 [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] boot 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:36.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:36.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:36.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:36.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:12:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:12:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:12:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: osd.5 [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] boot 2024-04-26T21:12:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:12:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:12:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:36.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:36.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:36.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:36.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:36.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:37.560 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:37 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi171 2024-04-26T21:12:37.561 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:37 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-26T21:12:37.561 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:37 smithi129 ceph-mon[29237]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:12:37.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:37 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi171 2024-04-26T21:12:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:37 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-26T21:12:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:37 smithi150 ceph-mon[32290]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:12:37.671 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:37 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi171 2024-04-26T21:12:37.671 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:37 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-26T21:12:37.672 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:37 smithi171 ceph-mon[33478]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.101+0000 7f175cf76700 1 -- 172.21.15.129:0/3885468129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17581050f0 msgr2=0x7f175810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.101+0000 7f175cf76700 1 --2- 172.21.15.129:0/3885468129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17581050f0 0x7f175810b9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f175000b0a0 tx=0x7f175000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.101+0000 7f175cf76700 1 -- 172.21.15.129:0/3885468129 shutdown_connections 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.101+0000 7f175cf76700 1 --2- 172.21.15.129:0/3885468129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17581050f0 0x7f175810b9a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.101+0000 7f175cf76700 1 --2- 172.21.15.129:0/3885468129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1758101770 0x7f1758101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.101+0000 7f175cf76700 1 --2- 172.21.15.129:0/3885468129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1758104750 0x7f1758104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.101+0000 7f175cf76700 1 -- 172.21.15.129:0/3885468129 >> 172.21.15.129:0/3885468129 conn(0x7f17580fd0d0 msgr2=0x7f17580ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.101+0000 7f175cf76700 1 -- 172.21.15.129:0/3885468129 shutdown_connections 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.101+0000 7f175cf76700 1 -- 172.21.15.129:0/3885468129 wait complete. 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175cf76700 1 Processor -- start 2024-04-26T21:12:38.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175cf76700 1 -- start start 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175cf76700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1758101770 0x7f1758199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175cf76700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1758104750 0x7f1758199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175cf76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581050f0 0x7f17581a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175cf76700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f175810ef20 con 0x7f1758104750 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175cf76700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f175810eda0 con 0x7f17581050f0 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175cf76700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f175810f0a0 con 0x7f1758101770 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1755d9b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1758104750 0x7f1758199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1755d9b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1758104750 0x7f1758199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49496/0 (socket says 172.21.15.129:49496) 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1755d9b700 1 -- 172.21.15.129:0/1007506213 learned_addr learned my addr 172.21.15.129:0/1007506213 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175659c700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1758101770 0x7f1758199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1755d9b700 1 -- 172.21.15.129:0/1007506213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1758101770 msgr2=0x7f1758199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1755d9b700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1758101770 0x7f1758199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1755d9b700 1 -- 172.21.15.129:0/1007506213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581050f0 msgr2=0x7f17581a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1756d9d700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581050f0 0x7f17581a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1755d9b700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581050f0 0x7f17581a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1755d9b700 1 -- 172.21.15.129:0/1007506213 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17581a1170 con 0x7f1758104750 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1756d9d700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581050f0 0x7f17581a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f1755d9b700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1758104750 0x7f1758199a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f174c00d980 tx=0x7f174c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f17477fe700 1 -- 172.21.15.129:0/1007506213 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c016070 con 0x7f1758104750 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17581a1460 con 0x7f1758104750 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.102+0000 7f17477fe700 1 -- 172.21.15.129:0/1007506213 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f174c00de70 con 0x7f1758104750 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.103+0000 7f17477fe700 1 -- 172.21.15.129:0/1007506213 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c00f3c0 con 0x7f1758104750 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.103+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17581a19a0 con 0x7f1758104750 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.104+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f175806a110 con 0x7f1758104750 2024-04-26T21:12:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.105+0000 7f17477fe700 1 -- 172.21.15.129:0/1007506213 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f174c019070 con 0x7f1758104750 2024-04-26T21:12:38.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.105+0000 7f17477fe700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17400a3c40 0x7f17400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.105+0000 7f175659c700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17400a3c40 0x7f17400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.105+0000 7f17477fe700 1 -- 172.21.15.129:0/1007506213 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f174c0cf090 con 0x7f1758104750 2024-04-26T21:12:38.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.106+0000 7f175659c700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17400a3c40 0x7f17400a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f175806b990 tx=0x7f174800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:38.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.110+0000 7f17477fe700 1 -- 172.21.15.129:0/1007506213 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f174c09c910 con 0x7f1758104750 2024-04-26T21:12:38.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:38.322+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1758063860 con 0x7f17400a3c40 2024-04-26T21:12:38.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:38 smithi129 ceph-mon[29237]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:12:38.592 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:38 smithi171 ceph-mon[33478]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:12:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:38 smithi150 ceph-mon[32290]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:12:39.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:39 smithi129 ceph-mon[29237]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:12:39.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:39 smithi129 ceph-mon[29237]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:39.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:12:39.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:12:39.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:39 smithi150 ceph-mon[32290]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:12:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:39 smithi150 ceph-mon[32290]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:12:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:12:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:39.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:39 smithi171 ceph-mon[33478]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:12:39.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:39 smithi171 ceph-mon[33478]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:39.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:12:39.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:12:39.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:40.556 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:40 smithi129 ceph-mon[29237]: pgmap v144: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:12:40.556 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:40 smithi129 ceph-mon[29237]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-26T21:12:40.556 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:40 smithi129 ceph-mon[29237]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:12:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:40 smithi150 ceph-mon[32290]: pgmap v144: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:12:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:40 smithi150 ceph-mon[32290]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-26T21:12:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:40 smithi150 ceph-mon[32290]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:12:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:40 smithi171 ceph-mon[33478]: pgmap v144: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:12:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:40 smithi171 ceph-mon[33478]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-26T21:12:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:40 smithi171 ceph-mon[33478]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:12:41.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:41 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2160626480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50c07cc9-ad4e-4283-9a5a-302eb0a948de"}]: dispatch 2024-04-26T21:12:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:41 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2160626480' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50c07cc9-ad4e-4283-9a5a-302eb0a948de"}]': finished 2024-04-26T21:12:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:41 smithi150 ceph-mon[32290]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:12:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:41 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:12:41.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:41 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2160626480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50c07cc9-ad4e-4283-9a5a-302eb0a948de"}]: dispatch 2024-04-26T21:12:41.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:41 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2160626480' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50c07cc9-ad4e-4283-9a5a-302eb0a948de"}]': finished 2024-04-26T21:12:41.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:41 smithi129 ceph-mon[29237]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:12:41.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:41 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:12:41.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:41 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2160626480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50c07cc9-ad4e-4283-9a5a-302eb0a948de"}]: dispatch 2024-04-26T21:12:41.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:41 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2160626480' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50c07cc9-ad4e-4283-9a5a-302eb0a948de"}]': finished 2024-04-26T21:12:41.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:41 smithi171 ceph-mon[33478]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:12:41.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:41 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:12:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:42 smithi150 ceph-mon[32290]: pgmap v147: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:12:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:42 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2361174386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:12:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:42 smithi129 ceph-mon[29237]: pgmap v147: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:12:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:42 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2361174386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:12:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:42 smithi171 ceph-mon[33478]: pgmap v147: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:12:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:42 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2361174386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:12:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:43 smithi150 ceph-mon[32290]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:12:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:43 smithi150 ceph-mon[32290]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-26T21:12:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:43 smithi150 ceph-mon[32290]: Cluster is now healthy 2024-04-26T21:12:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:43 smithi129 ceph-mon[29237]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:12:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:43 smithi129 ceph-mon[29237]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-26T21:12:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:43 smithi129 ceph-mon[29237]: Cluster is now healthy 2024-04-26T21:12:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:43 smithi171 ceph-mon[33478]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:12:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:43 smithi171 ceph-mon[33478]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-26T21:12:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:43 smithi171 ceph-mon[33478]: Cluster is now healthy 2024-04-26T21:12:46.464 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:46 smithi129 ceph-mon[29237]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-26T21:12:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:46 smithi150 ceph-mon[32290]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-26T21:12:46.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:46 smithi171 ceph-mon[33478]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-26T21:12:48.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:48 smithi171 ceph-mon[33478]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:12:48.615 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:48 smithi129 ceph-mon[29237]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:12:48.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:48 smithi150 ceph-mon[32290]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:12:49.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:49 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:12:49.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:49 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:49.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:49 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:12:49.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:49 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:49.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:49 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:12:49.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:49 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:50.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:50 smithi129 ceph-mon[29237]: Deploying daemon osd.6 on smithi129 2024-04-26T21:12:50.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:50 smithi129 ceph-mon[29237]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:12:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:50 smithi150 ceph-mon[32290]: Deploying daemon osd.6 on smithi129 2024-04-26T21:12:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:50 smithi150 ceph-mon[32290]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:12:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:50 smithi171 ceph-mon[33478]: Deploying daemon osd.6 on smithi129 2024-04-26T21:12:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:50 smithi171 ceph-mon[33478]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:12:52.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:52 smithi129 ceph-mon[29237]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-26T21:12:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:52 smithi150 ceph-mon[32290]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-26T21:12:52.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:52 smithi171 ceph-mon[33478]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-26T21:12:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:54 smithi150 ceph-mon[32290]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-26T21:12:54.652 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:54 smithi129 ceph-mon[29237]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-26T21:12:54.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:54 smithi171 ceph-mon[33478]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-26T21:12:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:56 smithi150 ceph-mon[32290]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:12:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:56 smithi129 ceph-mon[29237]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:12:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:56 smithi171 ceph-mon[33478]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:12:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:12:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:58.577 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:12:58 smithi129 ceph-mon[29237]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:12:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:12:58 smithi150 ceph-mon[32290]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:12:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:12:58 smithi171 ceph-mon[33478]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:12:59.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.036+0000 7f17477fe700 1 -- 172.21.15.129:0/1007506213 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1758063860 con 0x7f17400a3c40 2024-04-26T21:12:59.037 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 6 on host 'smithi129' 2024-04-26T21:12:59.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17400a3c40 msgr2=0x7f17400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17400a3c40 0x7f17400a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f175806b990 tx=0x7f174800a380 comp rx=0 tx=0).stop 2024-04-26T21:12:59.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1758104750 msgr2=0x7f1758199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1758104750 0x7f1758199a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f174c00d980 tx=0x7f174c00dc90 comp rx=0 tx=0).stop 2024-04-26T21:12:59.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 shutdown_connections 2024-04-26T21:12:59.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17400a3c40 0x7f17400a6100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581050f0 0x7f17581a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1758101770 0x7f1758199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 --2- 172.21.15.129:0/1007506213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1758104750 0x7f1758199a50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 >> 172.21.15.129:0/1007506213 conn(0x7f17580fd0d0 msgr2=0x7f1758107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:59.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 shutdown_connections 2024-04-26T21:12:59.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:12:59.038+0000 7f175cf76700 1 -- 172.21.15.129:0/1007506213 wait complete. 2024-04-26T21:12:59.811 DEBUG:teuthology.orchestra.run.smithi129:osd.6> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T21:12:59.813 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_2... 2024-04-26T21:12:59.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:13:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:13:00.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:00 smithi150 ceph-mon[32290]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:13:00.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:00 smithi150 ceph-mon[32290]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:13:00.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:00 smithi129 ceph-mon[29237]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:13:00.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:00 smithi129 ceph-mon[29237]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:13:00.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:00 smithi171 ceph-mon[33478]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:13:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:00 smithi171 ceph-mon[33478]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:13:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:00.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:01.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:01 smithi129 ceph-mon[29237]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:13:01.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:01 smithi129 ceph-mon[29237]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:13:01.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:01 smithi129 ceph-mon[29237]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:13:01.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:01.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:13:01 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:13:01.042+0000 7f86c5dea700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:13:01.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:13:01 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:13:01.051+0000 7f86c1bdd700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:13:01.294 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:01 smithi150 ceph-mon[32290]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:13:01.294 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:01 smithi150 ceph-mon[32290]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:13:01.294 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:01 smithi150 ceph-mon[32290]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:13:01.294 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:01.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:01 smithi171 ceph-mon[33478]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:13:01.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:01 smithi171 ceph-mon[33478]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:13:01.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:01 smithi171 ceph-mon[33478]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:13:01.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:02.168 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:02 smithi150 ceph-mon[32290]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:13:02.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:02 smithi150 ceph-mon[32290]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:13:02.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:02 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:02.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:02 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:02.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:02 smithi150 ceph-mon[32290]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:13:02.169 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:13:02.169 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:13:02.169 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-26T21:13:02.169 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-26T21:13:02.169 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0140347 s, 747 MB/s 2024-04-26T21:13:02.169 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 2024-04-26T21:13:02.169 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-26T21:13:02.314 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:02 smithi129 ceph-mon[29237]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:13:02.314 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:02 smithi129 ceph-mon[29237]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:13:02.314 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:02 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:02.314 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:02 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:02.314 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:02 smithi129 ceph-mon[29237]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:13:02.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:02 smithi171 ceph-mon[33478]: from='osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:13:02.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:02 smithi171 ceph-mon[33478]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:13:02.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:02 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:02.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:02 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:02.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:02 smithi171 ceph-mon[33478]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:13:02.836 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-04-26T21:13:03.083 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:13:03.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:03 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:13:03.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:03 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:13:03.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:03 smithi150 ceph-mon[32290]: osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] boot 2024-04-26T21:13:03.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:03 smithi150 ceph-mon[32290]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:13:03.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:03 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:03.215 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:03 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:13:03.216 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:03 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:13:03.216 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:03 smithi129 ceph-mon[29237]: osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] boot 2024-04-26T21:13:03.216 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:03 smithi129 ceph-mon[29237]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:13:03.216 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:03 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:03.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:03 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:13:03.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:03 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:13:03.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:03 smithi171 ceph-mon[33478]: osd.6 [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] boot 2024-04-26T21:13:03.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:03 smithi171 ceph-mon[33478]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:13:03.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:03 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:13:04.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:13:04.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:04.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:04.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:04.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:04.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:04.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:04.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:04.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:04.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:04.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:13:04.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:04.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:04.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:04.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:04.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:04.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:04.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:04.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:04.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:04.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:13:04.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:04.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:04.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:04.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.795+0000 7f7f5ab08700 1 -- 172.21.15.150:0/2603975280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 msgr2=0x7f7f5410a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.795+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/2603975280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 0x7f7f5410a7c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00b0d0 tx=0x7f7f4c00b3e0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.795+0000 7f7f5ab08700 1 -- 172.21.15.150:0/2603975280 shutdown_connections 2024-04-26T21:13:04.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.795+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/2603975280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 0x7f7f5410a7c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.795+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/2603975280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f54102370 0x7f7f54102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.795+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/2603975280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f54103570 0x7f7f541039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.795+0000 7f7f5ab08700 1 -- 172.21.15.150:0/2603975280 >> 172.21.15.150:0/2603975280 conn(0x7f7f540fd910 msgr2=0x7f7f540ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:04.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.795+0000 7f7f5ab08700 1 -- 172.21.15.150:0/2603975280 shutdown_connections 2024-04-26T21:13:04.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.795+0000 7f7f5ab08700 1 -- 172.21.15.150:0/2603975280 wait complete. 2024-04-26T21:13:04.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f5ab08700 1 Processor -- start 2024-04-26T21:13:04.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f5ab08700 1 -- start start 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f5ab08700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f54102370 0x7f7f5419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f5ab08700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f54103570 0x7f7f5419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f5ab08700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 0x7f7f541a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f5ab08700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5410de60 con 0x7f7f54103570 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f5ab08700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5410dce0 con 0x7f7f54103f10 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f5ab08700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5410dfe0 con 0x7f7f54102370 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f590a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 0x7f7f541a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f590a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 0x7f7f541a2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57260/0 (socket says 172.21.15.150:57260) 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f588a4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f54102370 0x7f7f5419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f590a5700 1 -- 172.21.15.150:0/480330379 learned_addr learned my addr 172.21.15.150:0/480330379 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.796+0000 7f7f53fff700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f54103570 0x7f7f5419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f590a5700 1 -- 172.21.15.150:0/480330379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f54102370 msgr2=0x7f7f5419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f590a5700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f54102370 0x7f7f5419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f590a5700 1 -- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f54103570 msgr2=0x7f7f5419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f590a5700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f54103570 0x7f7f5419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f590a5700 1 -- 172.21.15.150:0/480330379 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f541a3450 con 0x7f7f54103f10 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f590a5700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 0x7f7f541a2db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009040 tx=0x7f7f4c004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f51ffb700 1 -- 172.21.15.150:0/480330379 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c01f030 con 0x7f7f54103f10 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f51ffb700 1 -- 172.21.15.150:0/480330379 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f4c003a00 con 0x7f7f54103f10 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f541a36e0 con 0x7f7f54103f10 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f51ffb700 1 -- 172.21.15.150:0/480330379 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c004670 con 0x7f7f54103f10 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.797+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f541a3bb0 con 0x7f7f54103f10 2024-04-26T21:13:04.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.798+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f5406a110 con 0x7f7f54103f10 2024-04-26T21:13:04.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.799+0000 7f7f51ffb700 1 -- 172.21.15.150:0/480330379 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7f4c003bb0 con 0x7f7f54103f10 2024-04-26T21:13:04.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.800+0000 7f7f51ffb700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7f440a3c40 0x7f7f440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.800+0000 7f7f51ffb700 1 -- 172.21.15.150:0/480330379 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f7f4c00f020 con 0x7f7f54103f10 2024-04-26T21:13:04.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.800+0000 7f7f588a4700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7f440a3c40 0x7f7f440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.801+0000 7f7f588a4700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7f440a3c40 0x7f7f440a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f541033d0 tx=0x7f7f4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:04.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:04.804+0000 7f7f51ffb700 1 -- 172.21.15.150:0/480330379 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7f4c029020 con 0x7f7f54103f10 2024-04-26T21:13:05.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:05.011+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7f541092b0 con 0x7f7f440a3c40 2024-04-26T21:13:05.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:05 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi129 2024-04-26T21:13:05.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:05 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi129 to 5696M 2024-04-26T21:13:05.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:05 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:13:05.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:05 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:13:05.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:05 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:05.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:05 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi129 2024-04-26T21:13:05.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:05 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi129 to 5696M 2024-04-26T21:13:05.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:05 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:13:05.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:05 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:13:05.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:05 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:05.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:05 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi129 2024-04-26T21:13:05.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:05 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi129 to 5696M 2024-04-26T21:13:05.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:05 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:13:05.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:05 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:13:05.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:05 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:06.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:06 smithi150 ceph-mon[32290]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:13:06.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:06 smithi150 ceph-mon[32290]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:06.392 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:06 smithi171 ceph-mon[33478]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:13:06.392 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:06 smithi171 ceph-mon[33478]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:06.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:06 smithi129 ceph-mon[29237]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:13:06.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:06 smithi129 ceph-mon[29237]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:08 smithi150 ceph-mon[32290]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:08 smithi150 ceph-mon[32290]: from='client.? 172.21.15.150:0/2154035218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "189c2c39-0395-43a9-b250-a1e88d350428"}]: dispatch 2024-04-26T21:13:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:08 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "189c2c39-0395-43a9-b250-a1e88d350428"}]: dispatch 2024-04-26T21:13:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:08 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "189c2c39-0395-43a9-b250-a1e88d350428"}]': finished 2024-04-26T21:13:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:08 smithi150 ceph-mon[32290]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:13:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:08 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:08.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:08 smithi129 ceph-mon[29237]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:08.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:08 smithi129 ceph-mon[29237]: from='client.? 172.21.15.150:0/2154035218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "189c2c39-0395-43a9-b250-a1e88d350428"}]: dispatch 2024-04-26T21:13:08.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:08 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "189c2c39-0395-43a9-b250-a1e88d350428"}]: dispatch 2024-04-26T21:13:08.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:08 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "189c2c39-0395-43a9-b250-a1e88d350428"}]': finished 2024-04-26T21:13:08.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:08 smithi129 ceph-mon[29237]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:13:08.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:08 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:08.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:08 smithi171 ceph-mon[33478]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:08 smithi171 ceph-mon[33478]: from='client.? 172.21.15.150:0/2154035218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "189c2c39-0395-43a9-b250-a1e88d350428"}]: dispatch 2024-04-26T21:13:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:08 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "189c2c39-0395-43a9-b250-a1e88d350428"}]: dispatch 2024-04-26T21:13:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:08 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "189c2c39-0395-43a9-b250-a1e88d350428"}]': finished 2024-04-26T21:13:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:08 smithi171 ceph-mon[33478]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:13:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:08 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:09.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:09 smithi150 ceph-mon[32290]: from='client.? 172.21.15.150:0/1622423181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:13:09.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:09 smithi129 ceph-mon[29237]: from='client.? 172.21.15.150:0/1622423181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:13:09.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:09 smithi171 ceph-mon[33478]: from='client.? 172.21.15.150:0/1622423181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:13:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:10 smithi150 ceph-mon[32290]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:10 smithi129 ceph-mon[29237]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:10 smithi171 ceph-mon[33478]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:12 smithi150 ceph-mon[32290]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:12 smithi129 ceph-mon[29237]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:12.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:12 smithi171 ceph-mon[33478]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:14.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:14 smithi150 ceph-mon[32290]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:14 smithi129 ceph-mon[29237]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:14 smithi171 ceph-mon[33478]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:15.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:13:15.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:13:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:13:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:16 smithi150 ceph-mon[32290]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:16 smithi150 ceph-mon[32290]: Deploying daemon osd.7 on smithi150 2024-04-26T21:13:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:16 smithi129 ceph-mon[29237]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:16 smithi129 ceph-mon[29237]: Deploying daemon osd.7 on smithi150 2024-04-26T21:13:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:16 smithi171 ceph-mon[33478]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:16 smithi171 ceph-mon[33478]: Deploying daemon osd.7 on smithi150 2024-04-26T21:13:18.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:18 smithi150 ceph-mon[32290]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:18.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:18 smithi129 ceph-mon[29237]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:18.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:18 smithi171 ceph-mon[33478]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:20.604 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:20 smithi150 ceph-mon[32290]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:20.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:20 smithi129 ceph-mon[29237]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:20.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:20 smithi171 ceph-mon[33478]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:22.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:22.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:22.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:22.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:21 smithi150 ceph-mon[32290]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:22.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:22.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:22.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:22.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:21 smithi129 ceph-mon[29237]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:22.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:22.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:22.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:22.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:21 smithi171 ceph-mon[33478]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:24.364 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:24 smithi150 ceph-mon[32290]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:24.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.427+0000 7f7f51ffb700 1 -- 172.21.15.150:0/480330379 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7f541092b0 con 0x7f7f440a3c40 2024-04-26T21:13:24.429 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-04-26T21:13:24.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7f440a3c40 msgr2=0x7f7f440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:24.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7f440a3c40 0x7f7f440a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f541033d0 tx=0x7f7f4800b410 comp rx=0 tx=0).stop 2024-04-26T21:13:24.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 msgr2=0x7f7f541a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:24.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 0x7f7f541a2db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009040 tx=0x7f7f4c004d50 comp rx=0 tx=0).stop 2024-04-26T21:13:24.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 shutdown_connections 2024-04-26T21:13:24.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7f440a3c40 0x7f7f440a6100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:24.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54103f10 0x7f7f541a2db0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:24.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f54102370 0x7f7f5419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:24.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 --2- 172.21.15.150:0/480330379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f54103570 0x7f7f5419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:24.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 >> 172.21.15.150:0/480330379 conn(0x7f7f540fd910 msgr2=0x7f7f54108ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:24.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 shutdown_connections 2024-04-26T21:13:24.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:13:24.429+0000 7f7f5ab08700 1 -- 172.21.15.150:0/480330379 wait complete. 2024-04-26T21:13:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:24 smithi129 ceph-mon[29237]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:24 smithi171 ceph-mon[33478]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:25.061 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T21:13:25.063 INFO:tasks.cephadm:Deploying osd.8 on smithi171 with /dev/vg_nvme/lv_2... 2024-04-26T21:13:25.063 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:13:25.313 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:13:25.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:25 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:25 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:25 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:25 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:25 smithi150 ceph-mon[32290]: from='osd.7 [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:13:25.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:25 smithi150 ceph-mon[32290]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:13:25.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:25 smithi150 ceph-mon[32290]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:25 smithi129 ceph-mon[29237]: from='osd.7 [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:13:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:25 smithi129 ceph-mon[29237]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:13:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:25 smithi129 ceph-mon[29237]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:25.726 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:25 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.726 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:25 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.726 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:25 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.726 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:25 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:25.726 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:25 smithi171 ceph-mon[33478]: from='osd.7 [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:13:25.726 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:25 smithi171 ceph-mon[33478]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:13:25.726 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:25 smithi171 ceph-mon[33478]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:26 smithi171 ceph-mon[33478]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:13:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:26 smithi171 ceph-mon[33478]: from='osd.7 [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:13:26.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:26 smithi171 ceph-mon[33478]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:13:26.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:26.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:26 smithi171 ceph-mon[33478]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:13:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:26 smithi150 ceph-mon[32290]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:13:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:26 smithi150 ceph-mon[32290]: from='osd.7 [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:13:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:26 smithi150 ceph-mon[32290]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:13:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:26 smithi150 ceph-mon[32290]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:13:26.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:26 smithi129 ceph-mon[29237]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:13:26.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:26 smithi129 ceph-mon[29237]: from='osd.7 [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:13:26.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:26 smithi129 ceph-mon[29237]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:13:26.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:26.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:26 smithi129 ceph-mon[29237]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:13:27.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:13:27.080+0000 7f3ec663d700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:13:27.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:13:27.092+0000 7f3ebe428700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:13:27.520 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:13:27.520 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:13:27.521 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-26T21:13:27.521 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-26T21:13:27.521 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011615 s, 903 MB/s 2024-04-26T21:13:27.521 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-26T21:13:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-mon[32290]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:13:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-mon[32290]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:13:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-mon[32290]: from='osd.7 ' entity='osd.7' 2024-04-26T21:13:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-mon[32290]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-mon[32290]: osd.7 [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] boot 2024-04-26T21:13:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-mon[32290]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:13:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:27 smithi129 ceph-mon[29237]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:13:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:27 smithi129 ceph-mon[29237]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:13:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:27 smithi129 ceph-mon[29237]: from='osd.7 ' entity='osd.7' 2024-04-26T21:13:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:27 smithi129 ceph-mon[29237]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:27 smithi129 ceph-mon[29237]: osd.7 [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] boot 2024-04-26T21:13:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:27 smithi129 ceph-mon[29237]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:13:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:27 smithi171 ceph-mon[33478]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:13:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:27 smithi171 ceph-mon[33478]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:13:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:27 smithi171 ceph-mon[33478]: from='osd.7 ' entity='osd.7' 2024-04-26T21:13:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:27 smithi171 ceph-mon[33478]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:13:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:27 smithi171 ceph-mon[33478]: osd.7 [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] boot 2024-04-26T21:13:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:27 smithi171 ceph-mon[33478]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:13:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:13:28.173 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2024-04-26T21:13:28.419 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:13:28.808 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:13:28.808 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:13:28.808 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi150 2024-04-26T21:13:28.808 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.808 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:28 smithi171 ceph-mon[33478]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:13:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:13:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:13:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi150 2024-04-26T21:13:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:28.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:28 smithi129 ceph-mon[29237]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:13:29.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:13:29.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:13:29.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi150 2024-04-26T21:13:29.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:29.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:29.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:28 smithi150 ceph-mon[32290]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:13:29.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:29 smithi171 ceph-mon[33478]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:29.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:29 smithi129 ceph-mon[29237]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:30.061 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 -- 172.21.15.171:0/811873651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 msgr2=0x7f31a81038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 --2- 172.21.15.171:0/811873651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 0x7f31a81038a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009990 tx=0x7f31a4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 -- 172.21.15.171:0/811873651 shutdown_connections 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 --2- 172.21.15.171:0/811873651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8105020 0x7f31a810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 --2- 172.21.15.171:0/811873651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 0x7f31a81038a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 --2- 172.21.15.171:0/811873651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8104680 0x7f31a8104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 -- 172.21.15.171:0/811873651 >> 172.21.15.171:0/811873651 conn(0x7f31a80fea20 msgr2=0x7f31a8100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 -- 172.21.15.171:0/811873651 shutdown_connections 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 -- 172.21.15.171:0/811873651 wait complete. 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 Processor -- start 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.058+0000 7f31b047e700 1 -- start start 2024-04-26T21:13:30.062 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31b047e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 0x7f31a80771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31b047e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8104680 0x7f31a8075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31b047e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8105020 0x7f31a8075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31b047e700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31a810ef00 con 0x7f31a8105020 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31b047e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31a810ed80 con 0x7f31a8104680 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31b047e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f31a810f080 con 0x7f31a8103480 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ae21a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 0x7f31a80771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ae21a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 0x7f31a80771c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:47644/0 (socket says 172.21.15.171:47644) 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ae21a700 1 -- 172.21.15.171:0/288412038 learned_addr learned my addr 172.21.15.171:0/288412038 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31aea1b700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8105020 0x7f31a8075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ada19700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8104680 0x7f31a8075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ae21a700 1 -- 172.21.15.171:0/288412038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8104680 msgr2=0x7f31a8075810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ae21a700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8104680 0x7f31a8075810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:30.063 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ae21a700 1 -- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8105020 msgr2=0x7f31a8075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:30.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ae21a700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8105020 0x7f31a8075d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:30.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ae21a700 1 -- 172.21.15.171:0/288412038 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a8076600 con 0x7f31a8103480 2024-04-26T21:13:30.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31ae21a700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 0x7f31a80771c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f31a40036f0 tx=0x7f31a4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:30.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f319f7fe700 1 -- 172.21.15.171:0/288412038 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4027070 con 0x7f31a8103480 2024-04-26T21:13:30.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a81a36b0 con 0x7f31a8103480 2024-04-26T21:13:30.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f319f7fe700 1 -- 172.21.15.171:0/288412038 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a400bd90 con 0x7f31a8103480 2024-04-26T21:13:30.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.059+0000 7f319f7fe700 1 -- 172.21.15.171:0/288412038 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4022d40 con 0x7f31a8103480 2024-04-26T21:13:30.064 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.060+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a81a3b00 con 0x7f31a8103480 2024-04-26T21:13:30.065 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.060+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a806a110 con 0x7f31a8103480 2024-04-26T21:13:30.065 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.061+0000 7f319f7fe700 1 -- 172.21.15.171:0/288412038 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f31a402a070 con 0x7f31a8103480 2024-04-26T21:13:30.065 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.062+0000 7f319f7fe700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:30.065 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.062+0000 7f319f7fe700 1 -- 172.21.15.171:0/288412038 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f31a40d0a50 con 0x7f31a8103480 2024-04-26T21:13:30.065 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.062+0000 7f31ada19700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:30.065 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.063+0000 7f31ada19700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3198005950 tx=0x7f319800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:30.068 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.067+0000 7f319f7fe700 1 -- 172.21.15.171:0/288412038 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f31a4013070 con 0x7f31a8103480 2024-04-26T21:13:30.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:29 smithi150 ceph-mon[32290]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:30.274 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:30.272+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f31a8063860 con 0x7f31940a3c40 2024-04-26T21:13:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:30 smithi129 ceph-mon[29237]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:13:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:30 smithi129 ceph-mon[29237]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:13:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:13:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:13:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:30.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:30 smithi129 ceph-mon[29237]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:13:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:30 smithi171 ceph-mon[33478]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:13:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:30 smithi171 ceph-mon[33478]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:13:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:13:30.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:13:30.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:30.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:30 smithi171 ceph-mon[33478]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:13:31.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:30 smithi150 ceph-mon[32290]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:13:31.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:30 smithi150 ceph-mon[32290]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:13:31.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:13:31.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:13:31.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:31.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:30 smithi150 ceph-mon[32290]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:13:31.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:31 smithi129 ceph-mon[29237]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:31.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:31 smithi129 ceph-mon[29237]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:13:31.979 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:31 smithi171 ceph-mon[33478]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:31.979 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:31 smithi171 ceph-mon[33478]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:13:32.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:31 smithi150 ceph-mon[32290]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:32.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:31 smithi150 ceph-mon[32290]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:13:33.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:33 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/636077322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4382cd2f-fd00-45ef-9780-73271a167aa8"}]: dispatch 2024-04-26T21:13:33.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:33 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4382cd2f-fd00-45ef-9780-73271a167aa8"}]: dispatch 2024-04-26T21:13:33.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:33 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4382cd2f-fd00-45ef-9780-73271a167aa8"}]': finished 2024-04-26T21:13:33.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:33 smithi129 ceph-mon[29237]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:13:33.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:33.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:33 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/636077322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4382cd2f-fd00-45ef-9780-73271a167aa8"}]: dispatch 2024-04-26T21:13:33.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:33 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4382cd2f-fd00-45ef-9780-73271a167aa8"}]: dispatch 2024-04-26T21:13:33.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:33 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4382cd2f-fd00-45ef-9780-73271a167aa8"}]': finished 2024-04-26T21:13:33.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:33 smithi171 ceph-mon[33478]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:13:33.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:33 smithi150 ceph-mon[32290]: from='client.? 172.21.15.171:0/636077322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4382cd2f-fd00-45ef-9780-73271a167aa8"}]: dispatch 2024-04-26T21:13:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:33 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4382cd2f-fd00-45ef-9780-73271a167aa8"}]: dispatch 2024-04-26T21:13:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:33 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4382cd2f-fd00-45ef-9780-73271a167aa8"}]': finished 2024-04-26T21:13:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:33 smithi150 ceph-mon[32290]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:13:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:34.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:34 smithi129 ceph-mon[29237]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:34.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:34 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/2366311778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:13:34.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:34 smithi171 ceph-mon[33478]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:34.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:34 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/2366311778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:13:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:34 smithi150 ceph-mon[32290]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:34 smithi150 ceph-mon[32290]: from='client.? 172.21.15.171:0/2366311778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:13:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:36 smithi150 ceph-mon[32290]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:36.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:36 smithi129 ceph-mon[29237]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:36 smithi171 ceph-mon[33478]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:38.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:38 smithi171 ceph-mon[33478]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:38 smithi150 ceph-mon[32290]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:38.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:38 smithi129 ceph-mon[29237]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:40.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:40 smithi171 ceph-mon[33478]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:40 smithi150 ceph-mon[32290]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:40.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:40 smithi129 ceph-mon[29237]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:41.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:41 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:13:41.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:41 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:41 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:13:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:41 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:41.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:41 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:13:41.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:41 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:42.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:42 smithi171 ceph-mon[33478]: Deploying daemon osd.8 on smithi171 2024-04-26T21:13:42.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:42 smithi171 ceph-mon[33478]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:42.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:42 smithi150 ceph-mon[32290]: Deploying daemon osd.8 on smithi171 2024-04-26T21:13:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:42 smithi150 ceph-mon[32290]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:42 smithi129 ceph-mon[29237]: Deploying daemon osd.8 on smithi171 2024-04-26T21:13:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:42 smithi129 ceph-mon[29237]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:44.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:44 smithi171 ceph-mon[33478]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:44.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:44 smithi129 ceph-mon[29237]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:44.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:44 smithi150 ceph-mon[32290]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:46 smithi150 ceph-mon[32290]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:46.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:46 smithi129 ceph-mon[29237]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:46.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:46 smithi171 ceph-mon[33478]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:47.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:47.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:47.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:47.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:47 smithi129 ceph-mon[29237]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:47 smithi171 ceph-mon[33478]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:47 smithi150 ceph-mon[32290]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.394+0000 7f319f7fe700 1 -- 172.21.15.171:0/288412038 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f31a8063860 con 0x7f31940a3c40 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 8 on host 'smithi171' 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 msgr2=0x7f31940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3198005950 tx=0x7f319800b410 comp rx=0 tx=0).stop 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 msgr2=0x7f31a80771c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 0x7f31a80771c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f31a40036f0 tx=0x7f31a4009580 comp rx=0 tx=0).stop 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 shutdown_connections 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8105020 0x7f31a8075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8103480 0x7f31a80771c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 --2- 172.21.15.171:0/288412038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8104680 0x7f31a8075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 >> 172.21.15.171:0/288412038 conn(0x7f31a80fea20 msgr2=0x7f31a8107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 shutdown_connections 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:13:50.396+0000 7f31b047e700 1 -- 172.21.15.171:0/288412038 wait complete. 2024-04-26T21:13:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:50 smithi150 ceph-mon[32290]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:50 smithi150 ceph-mon[32290]: from='osd.8 [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:13:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:50 smithi150 ceph-mon[32290]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:13:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:50 smithi129 ceph-mon[29237]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:50.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:50 smithi129 ceph-mon[29237]: from='osd.8 [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:13:50.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:50 smithi129 ceph-mon[29237]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:13:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:50 smithi171 ceph-mon[33478]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:50.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:50.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:50 smithi171 ceph-mon[33478]: from='osd.8 [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:13:50.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:50 smithi171 ceph-mon[33478]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:13:51.085 DEBUG:teuthology.orchestra.run.smithi171:osd.8> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T21:13:51.088 INFO:tasks.cephadm:Deploying osd.9 on smithi129 with /dev/vg_nvme/lv_1... 2024-04-26T21:13:51.088 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:13:51.336 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:13:51.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:51 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:51.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:51 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:51.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:51 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:51.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:51 smithi129 ceph-mon[29237]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:13:51.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:51 smithi129 ceph-mon[29237]: from='osd.8 [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:13:51.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:51 smithi129 ceph-mon[29237]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:13:51.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:51 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:51.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:51 smithi129 ceph-mon[29237]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:13:51.458 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:51 smithi129 ceph-mon[29237]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:51.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:51.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:51.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:51.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-mon[33478]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:13:51.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-mon[33478]: from='osd.8 [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:13:51.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-mon[33478]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:13:51.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:51.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-mon[33478]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:13:51.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-mon[33478]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:51 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:51 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:51 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:51 smithi150 ceph-mon[32290]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:13:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:51 smithi150 ceph-mon[32290]: from='osd.8 [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:13:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:51 smithi150 ceph-mon[32290]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:13:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:51 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:51 smithi150 ceph-mon[32290]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:13:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:51 smithi150 ceph-mon[32290]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:52.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:13:51.957+0000 7fe46a818700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T21:13:52.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:13:51 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:13:51.967+0000 7fe462603700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:13:52.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:52 smithi171 ceph-mon[33478]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:13:52.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:52 smithi171 ceph-mon[33478]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:13:52.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:52 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:52.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:52 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:52.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:52 smithi171 ceph-mon[33478]: from='osd.8 ' entity='osd.8' 2024-04-26T21:13:52.837 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:52 smithi129 ceph-mon[29237]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:13:52.837 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:52 smithi129 ceph-mon[29237]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:13:52.837 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:52 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:52.837 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:52 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:52.838 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:52 smithi129 ceph-mon[29237]: from='osd.8 ' entity='osd.8' 2024-04-26T21:13:52.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:52 smithi150 ceph-mon[32290]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:13:52.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:52 smithi150 ceph-mon[32290]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:13:52.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:52 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:52.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:52 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:52.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:52 smithi150 ceph-mon[32290]: from='osd.8 ' entity='osd.8' 2024-04-26T21:13:53.423 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:13:53.423 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:13:53.423 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-26T21:13:53.423 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-26T21:13:53.424 INFO:teuthology.orchestra.run.smithi129.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114603 s, 915 MB/s 2024-04-26T21:13:53.424 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-26T21:13:53.824 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:53 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:13:53.824 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:53 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:13:53.824 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:53 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:53.824 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:53 smithi129 ceph-mon[29237]: osd.8 [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] boot 2024-04-26T21:13:53.824 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:53 smithi129 ceph-mon[29237]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T21:13:53.824 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:53 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:53.824 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:53 smithi129 ceph-mon[29237]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:53.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:53 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:13:53.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:53 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:13:53.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:53 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:53.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:53 smithi171 ceph-mon[33478]: osd.8 [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] boot 2024-04-26T21:13:53.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:53 smithi171 ceph-mon[33478]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T21:13:53.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:53 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:53.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:53 smithi171 ceph-mon[33478]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:53.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:53 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:13:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:53 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:13:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:53 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:53 smithi150 ceph-mon[32290]: osd.8 [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] boot 2024-04-26T21:13:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:53 smithi150 ceph-mon[32290]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T21:13:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:53 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:13:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:53 smithi150 ceph-mon[32290]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:13:54.170 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_1 2024-04-26T21:13:54.413 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:13:54.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:54 smithi129 ceph-mon[29237]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:13:54.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:54 smithi171 ceph-mon[33478]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:13:54.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:54 smithi150 ceph-mon[32290]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi171 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.674 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:55.675 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:55.675 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.675 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:55 smithi129 ceph-mon[29237]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi171 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:55.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:55 smithi150 ceph-mon[32290]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi171 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:55.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:55.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:13:55.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:55 smithi171 ceph-mon[33478]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:13:56.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.181+0000 7f5f31ea2700 1 -- 172.21.15.129:0/1956410544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 msgr2=0x7f5f2c104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.181+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/1956410544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 0x7f5f2c104bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009990 tx=0x7f5f20009ca0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.181+0000 7f5f31ea2700 1 -- 172.21.15.129:0/1956410544 shutdown_connections 2024-04-26T21:13:56.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.181+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/1956410544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f2c105110 0x7f5f2c10b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.181+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/1956410544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f2c101790 0x7f5f2c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.181+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/1956410544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 0x7f5f2c104bd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.181+0000 7f5f31ea2700 1 -- 172.21.15.129:0/1956410544 >> 172.21.15.129:0/1956410544 conn(0x7f5f2c0fd0f0 msgr2=0x7f5f2c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:56.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.181+0000 7f5f31ea2700 1 -- 172.21.15.129:0/1956410544 shutdown_connections 2024-04-26T21:13:56.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.182+0000 7f5f31ea2700 1 -- 172.21.15.129:0/1956410544 wait complete. 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.182+0000 7f5f31ea2700 1 Processor -- start 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.182+0000 7f5f31ea2700 1 -- start start 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.182+0000 7f5f31ea2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f2c101790 0x7f5f2c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f31ea2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 0x7f5f2c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f31ea2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f2c105110 0x7f5f2c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f31ea2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c10efb0 con 0x7f5f2c104770 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f31ea2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c10ee30 con 0x7f5f2c101790 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f31ea2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c10f130 con 0x7f5f2c105110 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2affd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 0x7f5f2c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2affd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 0x7f5f2c199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45278/0 (socket says 172.21.15.129:45278) 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2affd700 1 -- 172.21.15.129:0/286091393 learned_addr learned my addr 172.21.15.129:0/286091393 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2b7fe700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f2c101790 0x7f5f2c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2bfff700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f2c105110 0x7f5f2c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2affd700 1 -- 172.21.15.129:0/286091393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f2c105110 msgr2=0x7f5f2c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2affd700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f2c105110 0x7f5f2c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2affd700 1 -- 172.21.15.129:0/286091393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f2c101790 msgr2=0x7f5f2c199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2affd700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f2c101790 0x7f5f2c199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2affd700 1 -- 172.21.15.129:0/286091393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c1a1260 con 0x7f5f2c104770 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2bfff700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f2c105110 0x7f5f2c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f2affd700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 0x7f5f2c199b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5f200036f0 tx=0x7f5f200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f28ff9700 1 -- 172.21.15.129:0/286091393 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20027070 con 0x7f5f2c104770 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c1a14f0 con 0x7f5f2c104770 2024-04-26T21:13:56.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c1a1a20 con 0x7f5f2c104770 2024-04-26T21:13:56.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f28ff9700 1 -- 172.21.15.129:0/286091393 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f20004080 con 0x7f5f2c104770 2024-04-26T21:13:56.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.183+0000 7f5f28ff9700 1 -- 172.21.15.129:0/286091393 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20022d50 con 0x7f5f2c104770 2024-04-26T21:13:56.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.185+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2c06a110 con 0x7f5f2c104770 2024-04-26T21:13:56.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.185+0000 7f5f28ff9700 1 -- 172.21.15.129:0/286091393 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5f2002a070 con 0x7f5f2c104770 2024-04-26T21:13:56.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.186+0000 7f5f28ff9700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f140a3cc0 0x7f5f140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.186+0000 7f5f2b7fe700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f140a3cc0 0x7f5f140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.186+0000 7f5f28ff9700 1 -- 172.21.15.129:0/286091393 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f5f200d07a0 con 0x7f5f2c104770 2024-04-26T21:13:56.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.186+0000 7f5f2b7fe700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f140a3cc0 0x7f5f140a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c06b970 tx=0x7f5f1c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.191+0000 7f5f28ff9700 1 -- 172.21.15.129:0/286091393 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5f20013070 con 0x7f5f2c104770 2024-04-26T21:13:56.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:13:56.399+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5f2c063860 con 0x7f5f140a3cc0 2024-04-26T21:13:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:56 smithi129 ceph-mon[29237]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:13:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:13:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:13:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:56.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:56 smithi150 ceph-mon[32290]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:13:56.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:13:56.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:13:56.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:56.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:56 smithi171 ceph-mon[33478]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:13:56.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:13:56.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:13:56.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:57.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:57 smithi150 ceph-mon[32290]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:13:57.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:57 smithi150 ceph-mon[32290]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:13:57.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:13:57 smithi150 ceph-mon[32290]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:13:57.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:57 smithi129 ceph-mon[29237]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:13:57.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:57 smithi129 ceph-mon[29237]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:13:57.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:13:57 smithi129 ceph-mon[29237]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:13:57.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:57 smithi171 ceph-mon[33478]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:13:57.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:57 smithi171 ceph-mon[33478]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:13:57.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:13:57 smithi171 ceph-mon[33478]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:00 smithi150 ceph-mon[32290]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T21:14:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:00 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3528667062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c961ec13-aa37-4a66-b902-f36e159198b0"}]: dispatch 2024-04-26T21:14:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:00 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3528667062' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c961ec13-aa37-4a66-b902-f36e159198b0"}]': finished 2024-04-26T21:14:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:00 smithi150 ceph-mon[32290]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T21:14:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:00 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3488795747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:14:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:00 smithi129 ceph-mon[29237]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T21:14:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:00 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3528667062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c961ec13-aa37-4a66-b902-f36e159198b0"}]: dispatch 2024-04-26T21:14:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:00 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3528667062' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c961ec13-aa37-4a66-b902-f36e159198b0"}]': finished 2024-04-26T21:14:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:00 smithi129 ceph-mon[29237]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T21:14:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:00 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3488795747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:14:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:00 smithi171 ceph-mon[33478]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T21:14:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:00 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3528667062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c961ec13-aa37-4a66-b902-f36e159198b0"}]: dispatch 2024-04-26T21:14:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:00 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3528667062' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c961ec13-aa37-4a66-b902-f36e159198b0"}]': finished 2024-04-26T21:14:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:00 smithi171 ceph-mon[33478]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T21:14:00.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:00.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:00 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3488795747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:14:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:02 smithi150 ceph-mon[32290]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:14:02.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:02 smithi129 ceph-mon[29237]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:14:02.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:02 smithi171 ceph-mon[33478]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:14:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:04 smithi150 ceph-mon[32290]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-26T21:14:04.638 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:04 smithi129 ceph-mon[29237]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-26T21:14:04.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:04 smithi171 ceph-mon[33478]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-26T21:14:06.436 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:06 smithi129 ceph-mon[29237]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-26T21:14:06.544 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:06 smithi171 ceph-mon[33478]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-26T21:14:06.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:06 smithi150 ceph-mon[32290]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-26T21:14:07.557 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:14:07.557 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:07 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:14:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:07 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:07.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:14:07.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:07 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:08.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:08 smithi129 ceph-mon[29237]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:14:08.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:08 smithi129 ceph-mon[29237]: Deploying daemon osd.9 on smithi129 2024-04-26T21:14:08.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:08 smithi150 ceph-mon[32290]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:14:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:08 smithi150 ceph-mon[32290]: Deploying daemon osd.9 on smithi129 2024-04-26T21:14:08.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:08 smithi171 ceph-mon[33478]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:14:08.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:08 smithi171 ceph-mon[33478]: Deploying daemon osd.9 on smithi129 2024-04-26T21:14:10.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:10 smithi129 ceph-mon[29237]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:10 smithi150 ceph-mon[32290]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:10 smithi171 ceph-mon[33478]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:12.428 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:12 smithi129 ceph-mon[29237]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:12.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:12 smithi150 ceph-mon[32290]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:12.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:12 smithi171 ceph-mon[33478]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:14.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:14 smithi129 ceph-mon[29237]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:14.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:14 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:14.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:14 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:14.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:14 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:14 smithi150 ceph-mon[32290]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:14 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:14 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:14 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:14 smithi171 ceph-mon[33478]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:14 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:14 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:14.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:14 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:16.434 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:16 smithi129 ceph-mon[29237]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:16.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:16 smithi150 ceph-mon[32290]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:16 smithi171 ceph-mon[33478]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:17.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.419+0000 7f5f28ff9700 1 -- 172.21.15.129:0/286091393 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5f2c063860 con 0x7f5f140a3cc0 2024-04-26T21:14:17.421 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 9 on host 'smithi129' 2024-04-26T21:14:17.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f140a3cc0 msgr2=0x7f5f140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f140a3cc0 0x7f5f140a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c06b970 tx=0x7f5f1c00b410 comp rx=0 tx=0).stop 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 msgr2=0x7f5f2c199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 0x7f5f2c199b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5f200036f0 tx=0x7f5f200047b0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 shutdown_connections 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f140a3cc0 0x7f5f140a6180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f2c105110 0x7f5f2c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f2c101790 0x7f5f2c199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 --2- 172.21.15.129:0/286091393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f2c104770 0x7f5f2c199b40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.421+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 >> 172.21.15.129:0/286091393 conn(0x7f5f2c0fd0f0 msgr2=0x7f5f2c107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.422+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 shutdown_connections 2024-04-26T21:14:17.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:14:17.422+0000 7f5f31ea2700 1 -- 172.21.15.129:0/286091393 wait complete. 2024-04-26T21:14:18.149 DEBUG:teuthology.orchestra.run.smithi129:osd.9> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T21:14:18.151 INFO:tasks.cephadm:Deploying osd.10 on smithi150 with /dev/vg_nvme/lv_1... 2024-04-26T21:14:18.151 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:14:18.400 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:14:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:18 smithi129 ceph-mon[29237]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:18 smithi129 ceph-mon[29237]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:14:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:18.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:18 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:18.522 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:18 smithi150 ceph-mon[32290]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:18.522 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:18 smithi150 ceph-mon[32290]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:14:18.522 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:18.522 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:18 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:18.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:18 smithi171 ceph-mon[33478]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:18.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:18 smithi171 ceph-mon[33478]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:14:18.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:18.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:18 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:19.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:19 smithi129 ceph-mon[29237]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:14:19.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:19 smithi129 ceph-mon[29237]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:14:19.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:19 smithi129 ceph-mon[29237]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:14:19.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:19 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:19.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:19 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:19.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:19 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:19.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:19 smithi150 ceph-mon[32290]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:14:19.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:19 smithi150 ceph-mon[32290]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:14:19.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:19 smithi150 ceph-mon[32290]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:14:19.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:19 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:19.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:19 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:19.565 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:19 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:19.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:19 smithi171 ceph-mon[33478]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:14:19.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:19 smithi171 ceph-mon[33478]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:14:19.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:19 smithi171 ceph-mon[33478]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-26T21:14:19.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:19 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:19.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:19 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:19.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:19 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:20.220 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:14:19 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:14:19.744+0000 7fb05feba700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T21:14:20.220 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:14:19 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:14:19.756+0000 7fb0574a4700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:14:20.479 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:14:20.479 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:14:20.479 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-26T21:14:20.479 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-26T21:14:20.479 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116203 s, 902 MB/s 2024-04-26T21:14:20.479 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-26T21:14:20.555 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:20 smithi129 ceph-mon[29237]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:20.555 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:20 smithi129 ceph-mon[29237]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:14:20.555 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:20 smithi129 ceph-mon[29237]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:14:20.555 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:20 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:20.555 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:20 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:20.555 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:20 smithi129 ceph-mon[29237]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' 2024-04-26T21:14:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:20 smithi150 ceph-mon[32290]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:20 smithi150 ceph-mon[32290]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:14:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:20 smithi150 ceph-mon[32290]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:14:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:20 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:20 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:20 smithi150 ceph-mon[32290]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' 2024-04-26T21:14:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:20 smithi171 ceph-mon[33478]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:20 smithi171 ceph-mon[33478]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-26T21:14:20.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:20 smithi171 ceph-mon[33478]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:14:20.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:20 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:20.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:20 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:20.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:20 smithi171 ceph-mon[33478]: from='osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119]' entity='osd.9' 2024-04-26T21:14:21.193 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-04-26T21:14:21.442 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:14:21.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:21 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:14:21.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:21 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:14:21.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:21.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:21 smithi150 ceph-mon[32290]: osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] boot 2024-04-26T21:14:21.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:21 smithi150 ceph-mon[32290]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T21:14:21.564 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:21.694 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:21 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:14:21.694 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:21 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:14:21.694 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:21.694 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:21 smithi129 ceph-mon[29237]: osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] boot 2024-04-26T21:14:21.694 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:21 smithi129 ceph-mon[29237]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T21:14:21.694 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:21.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:21 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:14:21.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:21 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:14:21.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:21.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:21 smithi171 ceph-mon[33478]: osd.9 [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] boot 2024-04-26T21:14:21.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:21 smithi171 ceph-mon[33478]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T21:14:21.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:14:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:22.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:22.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:22 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:22.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:22.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:22 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:22.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:22 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:23.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.271+0000 7fdbdcef2700 1 -- 172.21.15.150:0/336356726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 msgr2=0x7fdbd8108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.271+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/336356726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 0x7fdbd8108cd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009990 tx=0x7fdbcc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.271+0000 7fdbdcef2700 1 -- 172.21.15.150:0/336356726 shutdown_connections 2024-04-26T21:14:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.271+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/336356726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd8109210 0x7fdbd810b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.271+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/336356726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd8103fb0 0x7fdbd81063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.271+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/336356726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 0x7fdbd8108cd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.271+0000 7fdbdcef2700 1 -- 172.21.15.150:0/336356726 >> 172.21.15.150:0/336356726 conn(0x7fdbd80fd920 msgr2=0x7fdbd80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.271+0000 7fdbdcef2700 1 -- 172.21.15.150:0/336356726 shutdown_connections 2024-04-26T21:14:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.271+0000 7fdbdcef2700 1 -- 172.21.15.150:0/336356726 wait complete. 2024-04-26T21:14:23.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbdcef2700 1 Processor -- start 2024-04-26T21:14:23.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbdcef2700 1 -- start start 2024-04-26T21:14:23.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbdcef2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd8103fb0 0x7fdbd8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbdcef2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 0x7fdbd8199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbdcef2700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd8109210 0x7fdbd81a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbdcef2700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd810ef60 con 0x7fdbd8109210 2024-04-26T21:14:23.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbdcef2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd810ede0 con 0x7fdbd81068e0 2024-04-26T21:14:23.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbdcef2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd810f0e0 con 0x7fdbd8103fb0 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbd5d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 0x7fdbd8199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbd5d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 0x7fdbd8199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33558/0 (socket says 172.21.15.150:33558) 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.272+0000 7fdbd5d9b700 1 -- 172.21.15.150:0/3791690098 learned_addr learned my addr 172.21.15.150:0/3791690098 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbd6d9d700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd8109210 0x7fdbd81a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbd659c700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd8103fb0 0x7fdbd8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbd5d9b700 1 -- 172.21.15.150:0/3791690098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd8103fb0 msgr2=0x7fdbd8199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbd5d9b700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd8103fb0 0x7fdbd8199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbd5d9b700 1 -- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd8109210 msgr2=0x7fdbd81a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbd5d9b700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd8109210 0x7fdbd81a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbd5d9b700 1 -- 172.21.15.150:0/3791690098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd8101da0 con 0x7fdbd81068e0 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbd5d9b700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 0x7fdbd8199ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc006010 tx=0x7fdbcc0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbc77fe700 1 -- 172.21.15.150:0/3791690098 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc027070 con 0x7fdbd81068e0 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbc77fe700 1 -- 172.21.15.150:0/3791690098 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc004000 con 0x7fdbd81068e0 2024-04-26T21:14:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd8102030 con 0x7fdbd81068e0 2024-04-26T21:14:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbc77fe700 1 -- 172.21.15.150:0/3791690098 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc022c80 con 0x7fdbd81068e0 2024-04-26T21:14:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.273+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd8102590 con 0x7fdbd81068e0 2024-04-26T21:14:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.274+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd806a110 con 0x7fdbd81068e0 2024-04-26T21:14:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.275+0000 7fdbc77fe700 1 -- 172.21.15.150:0/3791690098 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdbcc02a070 con 0x7fdbd81068e0 2024-04-26T21:14:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.276+0000 7fdbc77fe700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdbc00a3c40 0x7fdbc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.276+0000 7fdbc77fe700 1 -- 172.21.15.150:0/3791690098 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fdbcc0d0b40 con 0x7fdbd81068e0 2024-04-26T21:14:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.276+0000 7fdbd659c700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdbc00a3c40 0x7fdbc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.277+0000 7fdbd659c700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdbc00a3c40 0x7fdbc00a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000a400 tx=0x7fdbd000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.281+0000 7fdbc77fe700 1 -- 172.21.15.150:0/3791690098 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdbcc05f060 con 0x7fdbd81068e0 2024-04-26T21:14:23.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:23.492+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdbd80fef90 con 0x7fdbc00a3c40 2024-04-26T21:14:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:23 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi129 2024-04-26T21:14:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:23 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi129 to 4272M 2024-04-26T21:14:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:23 smithi150 ceph-mon[32290]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T21:14:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:23 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi129 2024-04-26T21:14:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:23 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi129 to 4272M 2024-04-26T21:14:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:23 smithi129 ceph-mon[29237]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T21:14:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:23 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi129 2024-04-26T21:14:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:23 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi129 to 4272M 2024-04-26T21:14:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:23 smithi171 ceph-mon[33478]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T21:14:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:23.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:24 smithi150 ceph-mon[32290]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:14:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:14:24.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:24 smithi129 ceph-mon[29237]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:14:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:14:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:24 smithi171 ceph-mon[33478]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:14:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:14:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:25.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:25 smithi150 ceph-mon[32290]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:14:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:25 smithi129 ceph-mon[29237]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:14:25.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:25 smithi171 ceph-mon[33478]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:14:26.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:26 smithi150 ceph-mon[32290]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:26 smithi129 ceph-mon[29237]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:26 smithi171 ceph-mon[33478]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:27.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:27 smithi150 ceph-mon[32290]: from='client.? 172.21.15.150:0/1209676534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eeeec10-5f31-4453-99bb-b705a557e632"}]: dispatch 2024-04-26T21:14:27.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:27 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eeeec10-5f31-4453-99bb-b705a557e632"}]: dispatch 2024-04-26T21:14:27.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:27 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2eeeec10-5f31-4453-99bb-b705a557e632"}]': finished 2024-04-26T21:14:27.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:27 smithi150 ceph-mon[32290]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T21:14:27.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:27.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:27 smithi150 ceph-mon[32290]: from='client.? 172.21.15.150:0/470858563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:14:27.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:27 smithi129 ceph-mon[29237]: from='client.? 172.21.15.150:0/1209676534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eeeec10-5f31-4453-99bb-b705a557e632"}]: dispatch 2024-04-26T21:14:27.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:27 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eeeec10-5f31-4453-99bb-b705a557e632"}]: dispatch 2024-04-26T21:14:27.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:27 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2eeeec10-5f31-4453-99bb-b705a557e632"}]': finished 2024-04-26T21:14:27.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:27 smithi129 ceph-mon[29237]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T21:14:27.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:27.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:27 smithi129 ceph-mon[29237]: from='client.? 172.21.15.150:0/470858563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:14:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:27 smithi171 ceph-mon[33478]: from='client.? 172.21.15.150:0/1209676534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eeeec10-5f31-4453-99bb-b705a557e632"}]: dispatch 2024-04-26T21:14:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:27 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eeeec10-5f31-4453-99bb-b705a557e632"}]: dispatch 2024-04-26T21:14:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:27 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2eeeec10-5f31-4453-99bb-b705a557e632"}]': finished 2024-04-26T21:14:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:27 smithi171 ceph-mon[33478]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T21:14:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:27 smithi171 ceph-mon[33478]: from='client.? 172.21.15.150:0/470858563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:14:28.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:28 smithi150 ceph-mon[32290]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:28.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:28 smithi129 ceph-mon[29237]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:28 smithi171 ceph-mon[33478]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:30.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:30 smithi150 ceph-mon[32290]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:30.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:30 smithi129 ceph-mon[29237]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:30 smithi171 ceph-mon[33478]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:32 smithi150 ceph-mon[32290]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:32 smithi129 ceph-mon[29237]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:32.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:32 smithi171 ceph-mon[33478]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:34.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:34 smithi150 ceph-mon[32290]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:34.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:14:34.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:34 smithi129 ceph-mon[29237]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:14:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:34.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:34 smithi171 ceph-mon[33478]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:34.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:14:34.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:35.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:35 smithi150 ceph-mon[32290]: Deploying daemon osd.10 on smithi150 2024-04-26T21:14:35.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:35 smithi150 ceph-mon[32290]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:35.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:35 smithi129 ceph-mon[29237]: Deploying daemon osd.10 on smithi150 2024-04-26T21:14:35.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:35 smithi129 ceph-mon[29237]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:35.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:35 smithi171 ceph-mon[33478]: Deploying daemon osd.10 on smithi150 2024-04-26T21:14:35.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:35 smithi171 ceph-mon[33478]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:38.616 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:38 smithi150 ceph-mon[32290]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:38 smithi129 ceph-mon[29237]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:38 smithi171 ceph-mon[33478]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:40.614 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:40 smithi150 ceph-mon[32290]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:40.614 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:40 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:40.614 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:40 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:40.614 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:40 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:40.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:40 smithi129 ceph-mon[29237]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:40.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:40 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:40.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:40 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:40.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:40 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:40 smithi171 ceph-mon[33478]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:40 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:42.591 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:42 smithi150 ceph-mon[32290]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:42 smithi129 ceph-mon[29237]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:42 smithi171 ceph-mon[33478]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:43.709 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 10 on host 'smithi150' 2024-04-26T21:14:43.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.707+0000 7fdbc77fe700 1 -- 172.21.15.150:0/3791690098 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdbd80fef90 con 0x7fdbc00a3c40 2024-04-26T21:14:43.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdbc00a3c40 msgr2=0x7fdbc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:43.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdbc00a3c40 0x7fdbc00a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000a400 tx=0x7fdbd000d040 comp rx=0 tx=0).stop 2024-04-26T21:14:43.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 msgr2=0x7fdbd8199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 0x7fdbd8199ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc006010 tx=0x7fdbcc0047b0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 shutdown_connections 2024-04-26T21:14:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdbc00a3c40 0x7fdbc00a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbd8109210 0x7fdbd81a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd8103fb0 0x7fdbd8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 --2- 172.21.15.150:0/3791690098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd81068e0 0x7fdbd8199ba0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 >> 172.21.15.150:0/3791690098 conn(0x7fdbd80fd920 msgr2=0x7fdbd8104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 shutdown_connections 2024-04-26T21:14:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-26T21:14:43.710+0000 7fdbdcef2700 1 -- 172.21.15.150:0/3791690098 wait complete. 2024-04-26T21:14:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:44 smithi150 ceph-mon[32290]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:44 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:44 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:44 smithi150 ceph-mon[32290]: from='osd.10 [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:14:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:44 smithi150 ceph-mon[32290]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:14:44.426 DEBUG:teuthology.orchestra.run.smithi150:osd.10> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T21:14:44.429 INFO:tasks.cephadm:Deploying osd.11 on smithi171 with /dev/vg_nvme/lv_1... 2024-04-26T21:14:44.429 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:14:44.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:44 smithi129 ceph-mon[29237]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:44.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:44 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:44.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:44 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:44.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:44 smithi129 ceph-mon[29237]: from='osd.10 [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:14:44.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:44 smithi129 ceph-mon[29237]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:14:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:44 smithi171 ceph-mon[33478]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:44 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:44 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:44 smithi171 ceph-mon[33478]: from='osd.10 [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:14:44.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:44 smithi171 ceph-mon[33478]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:14:45.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:45.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:45.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-mon[32290]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:14:45.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-mon[32290]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:14:45.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:45.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-mon[32290]: from='osd.10 [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:14:45.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-mon[32290]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:14:45.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-mon[32290]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:45.812 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:45.812 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:45.812 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:45 smithi129 ceph-mon[29237]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:14:45.812 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:45 smithi129 ceph-mon[29237]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:14:45.812 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:45.812 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:45 smithi129 ceph-mon[29237]: from='osd.10 [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:14:45.812 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:45 smithi129 ceph-mon[29237]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:14:45.812 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:45 smithi129 ceph-mon[29237]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:45.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:45.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:45.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:45 smithi171 ceph-mon[33478]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:14:45.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:45 smithi171 ceph-mon[33478]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:14:45.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:45.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:45 smithi171 ceph-mon[33478]: from='osd.10 [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:14:45.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:45 smithi171 ceph-mon[33478]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-26T21:14:45.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:45 smithi171 ceph-mon[33478]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:46.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:14:45.706+0000 7f1e622e6700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T21:14:46.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:14:45 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:14:45.720+0000 7f1e5e0d9700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:14:46.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:46 smithi150 ceph-mon[32290]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:14:46.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:46 smithi150 ceph-mon[32290]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:14:46.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:46 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:46.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:46 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:46.910 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:14:46.910 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:14:46.910 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-26T21:14:46.910 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-26T21:14:46.911 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114085 s, 919 MB/s 2024-04-26T21:14:46.911 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 2024-04-26T21:14:46.911 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-26T21:14:46.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:46 smithi129 ceph-mon[29237]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:14:46.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:46 smithi129 ceph-mon[29237]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:14:46.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:46 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:46.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:46 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:46.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:46 smithi171 ceph-mon[33478]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-26T21:14:46.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:46 smithi171 ceph-mon[33478]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:14:46.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:46 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:46.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:46 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:47.708 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2024-04-26T21:14:47.950 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:14:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:47 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:14:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:47 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:14:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:47 smithi129 ceph-mon[29237]: osd.10 [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] boot 2024-04-26T21:14:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:47 smithi129 ceph-mon[29237]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T21:14:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:47 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:47 smithi129 ceph-mon[29237]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:47.989 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:47 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:14:47.989 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:47 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:14:47.989 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:47 smithi171 ceph-mon[33478]: osd.10 [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] boot 2024-04-26T21:14:47.989 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:47 smithi171 ceph-mon[33478]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T21:14:47.990 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:47 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:47.990 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:47 smithi171 ceph-mon[33478]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:48.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:47 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:14:48.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:47 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:14:48.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:47 smithi150 ceph-mon[32290]: osd.10 [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] boot 2024-04-26T21:14:48.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:47 smithi150 ceph-mon[32290]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T21:14:48.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:47 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:14:48.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:47 smithi150 ceph-mon[32290]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi150 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-26T21:14:48.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:48.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:48.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:48.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:48.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:48.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:48 smithi129 ceph-mon[29237]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:14:48.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi150 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:48.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:48.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:48.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:48 smithi171 ceph-mon[33478]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:14:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:14:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi150 2024-04-26T21:14:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:14:49.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-26T21:14:49.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:49.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:49.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:49.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:49.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:49.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:48 smithi150 ceph-mon[32290]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:14:49.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.831+0000 7f9f32195700 1 -- 172.21.15.171:0/2201558357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c102380 msgr2=0x7f9f2c1027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:49.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.831+0000 7f9f32195700 1 --2- 172.21.15.171:0/2201558357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c102380 0x7f9f2c1027a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009990 tx=0x7f9f1c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:14:49.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.831+0000 7f9f32195700 1 -- 172.21.15.171:0/2201558357 shutdown_connections 2024-04-26T21:14:49.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.831+0000 7f9f32195700 1 --2- 172.21.15.171:0/2201558357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f2c103f20 0x7f9f2c10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:49.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.831+0000 7f9f32195700 1 --2- 172.21.15.171:0/2201558357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c102380 0x7f9f2c1027a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:49.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.831+0000 7f9f32195700 1 --2- 172.21.15.171:0/2201558357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f2c103580 0x7f9f2c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:49.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.831+0000 7f9f32195700 1 -- 172.21.15.171:0/2201558357 >> 172.21.15.171:0/2201558357 conn(0x7f9f2c0fd920 msgr2=0x7f9f2c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:49.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.831+0000 7f9f32195700 1 -- 172.21.15.171:0/2201558357 shutdown_connections 2024-04-26T21:14:49.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.831+0000 7f9f32195700 1 -- 172.21.15.171:0/2201558357 wait complete. 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f32195700 1 Processor -- start 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f32195700 1 -- start start 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f32195700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f2c102380 0x7f9f2c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f32195700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c103580 0x7f9f2c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f32195700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f2c103f20 0x7f9f2c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f32195700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c10de70 con 0x7f9f2c102380 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f32195700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c10dcf0 con 0x7f9f2c103f20 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f32195700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c10dff0 con 0x7f9f2c103580 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c103580 0x7f9f2c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c103580 0x7f9f2c198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:39804/0 (socket says 172.21.15.171:39804) 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2affd700 1 -- 172.21.15.171:0/2444955345 learned_addr learned my addr 172.21.15.171:0/2444955345 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2b7fe700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f2c102380 0x7f9f2c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2bfff700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f2c103f20 0x7f9f2c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:49.835 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2affd700 1 -- 172.21.15.171:0/2444955345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f2c103f20 msgr2=0x7f9f2c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2affd700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f2c103f20 0x7f9f2c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2affd700 1 -- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f2c102380 msgr2=0x7f9f2c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2affd700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f2c102380 0x7f9f2c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.832+0000 7f9f2affd700 1 -- 172.21.15.171:0/2444955345 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c1a01f0 con 0x7f9f2c103580 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.833+0000 7f9f2affd700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c103580 0x7f9f2c198b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000d980 tx=0x7f9f2000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.833+0000 7f9f28ff9700 1 -- 172.21.15.171:0/2444955345 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20016070 con 0x7f9f2c103580 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.833+0000 7f9f28ff9700 1 -- 172.21.15.171:0/2444955345 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f2000de70 con 0x7f9f2c103580 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.833+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c1a04e0 con 0x7f9f2c103580 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.833+0000 7f9f28ff9700 1 -- 172.21.15.171:0/2444955345 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000f3c0 con 0x7f9f2c103580 2024-04-26T21:14:49.836 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.833+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c1a0a70 con 0x7f9f2c103580 2024-04-26T21:14:49.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.834+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2c06a110 con 0x7f9f2c103580 2024-04-26T21:14:49.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.835+0000 7f9f28ff9700 1 -- 172.21.15.171:0/2444955345 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9f20019070 con 0x7f9f2c103580 2024-04-26T21:14:49.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.836+0000 7f9f28ff9700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f140a3cc0 0x7f9f140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:49.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.836+0000 7f9f28ff9700 1 -- 172.21.15.171:0/2444955345 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f9f200cf840 con 0x7f9f2c103580 2024-04-26T21:14:49.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.836+0000 7f9f2b7fe700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f140a3cc0 0x7f9f140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:49.837 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.836+0000 7f9f2b7fe700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f140a3cc0 0x7f9f140a6180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00b5c0 tx=0x7f9f1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:49.840 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:49.839+0000 7f9f28ff9700 1 -- 172.21.15.171:0/2444955345 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9f2009c870 con 0x7f9f2c103580 2024-04-26T21:14:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:49 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:49 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:49 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:49 smithi129 ceph-mon[29237]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:14:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:49 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:49 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:49 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:49 smithi171 ceph-mon[33478]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:14:50.047 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:14:50.047+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9f2c063860 con 0x7f9f140a3cc0 2024-04-26T21:14:50.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:49 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:50.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:49 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:50.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:49 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:14:50.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:49 smithi150 ceph-mon[32290]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:14:50.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:50 smithi171 ceph-mon[33478]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:14:50.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:50 smithi171 ceph-mon[33478]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:14:50.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:14:50.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:14:50.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:51.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:50 smithi150 ceph-mon[32290]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:14:51.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:50 smithi150 ceph-mon[32290]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:14:51.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:14:51.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:14:51.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:51.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:50 smithi129 ceph-mon[29237]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:14:51.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:50 smithi129 ceph-mon[29237]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:14:51.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:14:51.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:14:51.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:51.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:51 smithi171 ceph-mon[33478]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:14:51.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:51 smithi171 ceph-mon[33478]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:14:52.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:51 smithi150 ceph-mon[32290]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:14:52.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:51 smithi150 ceph-mon[32290]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:14:52.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:51 smithi129 ceph-mon[29237]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:14:52.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:51 smithi129 ceph-mon[29237]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:14:53.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:52 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/844945445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae01372d-419f-4def-9fcd-3d49eb50b0e0"}]: dispatch 2024-04-26T21:14:53.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:52 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae01372d-419f-4def-9fcd-3d49eb50b0e0"}]: dispatch 2024-04-26T21:14:53.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:52 smithi129 ceph-mon[29237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae01372d-419f-4def-9fcd-3d49eb50b0e0"}]': finished 2024-04-26T21:14:53.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:52 smithi129 ceph-mon[29237]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T21:14:53.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:52 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:14:53.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:52 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/844945445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae01372d-419f-4def-9fcd-3d49eb50b0e0"}]: dispatch 2024-04-26T21:14:53.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:52 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae01372d-419f-4def-9fcd-3d49eb50b0e0"}]: dispatch 2024-04-26T21:14:53.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:52 smithi171 ceph-mon[33478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae01372d-419f-4def-9fcd-3d49eb50b0e0"}]': finished 2024-04-26T21:14:53.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:52 smithi171 ceph-mon[33478]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T21:14:53.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:52 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:14:53.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:52 smithi150 ceph-mon[32290]: from='client.? 172.21.15.171:0/844945445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae01372d-419f-4def-9fcd-3d49eb50b0e0"}]: dispatch 2024-04-26T21:14:53.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:52 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae01372d-419f-4def-9fcd-3d49eb50b0e0"}]: dispatch 2024-04-26T21:14:53.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:52 smithi150 ceph-mon[32290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae01372d-419f-4def-9fcd-3d49eb50b0e0"}]': finished 2024-04-26T21:14:53.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:52 smithi150 ceph-mon[32290]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T21:14:53.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:52 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:14:54.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:53 smithi129 ceph-mon[29237]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-26T21:14:54.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:53 smithi129 ceph-mon[29237]: from='client.? 172.21.15.171:0/2999928133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:14:54.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:53 smithi171 ceph-mon[33478]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-26T21:14:54.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:53 smithi171 ceph-mon[33478]: from='client.? 172.21.15.171:0/2999928133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:14:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:53 smithi150 ceph-mon[32290]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-26T21:14:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:53 smithi150 ceph-mon[32290]: from='client.? 172.21.15.171:0/2999928133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:14:56.585 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:56 smithi171 ceph-mon[33478]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:14:56.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:56 smithi150 ceph-mon[32290]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:14:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:56 smithi129 ceph-mon[29237]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:14:58.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:14:58 smithi171 ceph-mon[33478]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T21:14:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:14:58 smithi150 ceph-mon[32290]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T21:14:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:14:58 smithi129 ceph-mon[29237]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T21:15:00.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:00 smithi171 ceph-mon[33478]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-26T21:15:00.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:00 smithi150 ceph-mon[32290]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-26T21:15:00.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:00 smithi129 ceph-mon[29237]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-26T21:15:01.461 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:15:01.461 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:01.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:15:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:15:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:02.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:02 smithi171 ceph-mon[33478]: Deploying daemon osd.11 on smithi171 2024-04-26T21:15:02.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:02 smithi171 ceph-mon[33478]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:15:02.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:02 smithi150 ceph-mon[32290]: Deploying daemon osd.11 on smithi171 2024-04-26T21:15:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:02 smithi150 ceph-mon[32290]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:15:02.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:02 smithi129 ceph-mon[29237]: Deploying daemon osd.11 on smithi171 2024-04-26T21:15:02.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:02 smithi129 ceph-mon[29237]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:15:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:04 smithi150 ceph-mon[32290]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:04 smithi129 ceph-mon[29237]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:04.720 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:04 smithi171 ceph-mon[33478]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:06 smithi150 ceph-mon[32290]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:06.636 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:06 smithi171 ceph-mon[33478]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:06.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:06 smithi129 ceph-mon[29237]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:08 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:08 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:08 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:08 smithi150 ceph-mon[32290]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:08.390 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:08 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:08.390 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:08 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:08.391 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:08 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:08.391 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:08 smithi171 ceph-mon[33478]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:08.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:08 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:08.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:08 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:08.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:08 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:08.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:08 smithi129 ceph-mon[29237]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:10.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:10 smithi171 ceph-mon[33478]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:10.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:10.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:10.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:10.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:10.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:10 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:10 smithi150 ceph-mon[32290]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:10 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:10 smithi129 ceph-mon[29237]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:10 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:11.018 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.015+0000 7f9f28ff9700 1 -- 172.21.15.171:0/2444955345 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9f2c063860 con 0x7f9f140a3cc0 2024-04-26T21:15:11.018 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 11 on host 'smithi171' 2024-04-26T21:15:11.019 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f140a3cc0 msgr2=0x7f9f140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.019 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f140a3cc0 0x7f9f140a6180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00b5c0 tx=0x7f9f1c005fb0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c103580 msgr2=0x7f9f2c198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c103580 0x7f9f2c198b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000d980 tx=0x7f9f2000dc90 comp rx=0 tx=0).stop 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 shutdown_connections 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f140a3cc0 0x7f9f140a6180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f2c103f20 0x7f9f2c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f2c102380 0x7f9f2c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 --2- 172.21.15.171:0/2444955345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f2c103580 0x7f9f2c198b20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 >> 172.21.15.171:0/2444955345 conn(0x7f9f2c0fd920 msgr2=0x7f9f2c1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 shutdown_connections 2024-04-26T21:15:11.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:11.018+0000 7f9f32195700 1 -- 172.21.15.171:0/2444955345 wait complete. 2024-04-26T21:15:11.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:11 smithi150 ceph-mon[32290]: from='osd.11 [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:15:11.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:11 smithi150 ceph-mon[32290]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:15:11.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:11.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:11.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:11 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:11.668 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:11 smithi171 ceph-mon[33478]: from='osd.11 [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:15:11.668 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:11 smithi171 ceph-mon[33478]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:15:11.668 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:11.668 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:11.668 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:11 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:11.695 DEBUG:teuthology.orchestra.run.smithi171:osd.11> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T21:15:11.698 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T21:15:11.698 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T21:15:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:11 smithi129 ceph-mon[29237]: from='osd.11 [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:15:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:11 smithi129 ceph-mon[29237]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:15:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:11 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:11.945 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:15:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:12 smithi150 ceph-mon[32290]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:12 smithi150 ceph-mon[32290]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:15:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:12 smithi150 ceph-mon[32290]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:15:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:12 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:12 smithi150 ceph-mon[32290]: from='osd.11 [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:15:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:12 smithi150 ceph-mon[32290]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:15:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:12 smithi129 ceph-mon[29237]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:12 smithi129 ceph-mon[29237]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:15:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:12 smithi129 ceph-mon[29237]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:15:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:12 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:12 smithi129 ceph-mon[29237]: from='osd.11 [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:15:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:12 smithi129 ceph-mon[29237]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:15:12.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:12 smithi171 ceph-mon[33478]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:12.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:12 smithi171 ceph-mon[33478]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:15:12.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:12 smithi171 ceph-mon[33478]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:15:12.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:12 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:12.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:12 smithi171 ceph-mon[33478]: from='osd.11 [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:15:12.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:12 smithi171 ceph-mon[33478]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-26T21:15:12.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:15:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:15:12.330+0000 7ff70ade2700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T21:15:12.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:15:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:15:12.341+0000 7ff706bd5700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:15:13.376 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:13 smithi129 ceph-mon[29237]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:15:13.376 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:13 smithi129 ceph-mon[29237]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:15:13.376 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:13 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:13.376 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:13 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:13.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:13 smithi150 ceph-mon[32290]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:15:13.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:13 smithi150 ceph-mon[32290]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:15:13.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:13 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:13.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:13 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:13.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:13 smithi171 ceph-mon[33478]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-26T21:15:13.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:13 smithi171 ceph-mon[33478]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:15:13.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:13 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:13.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:13 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:13.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.741+0000 7f28a40b6700 1 -- 172.21.15.129:0/353821693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 msgr2=0x7f289c104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.741+0000 7f28a40b6700 1 --2- 172.21.15.129:0/353821693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 0x7f289c104ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28980099c0 tx=0x7f2898009cd0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.741+0000 7f28a40b6700 1 -- 172.21.15.129:0/353821693 shutdown_connections 2024-04-26T21:15:13.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.741+0000 7f28a40b6700 1 --2- 172.21.15.129:0/353821693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f289c105010 0x7f289c10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.741+0000 7f28a40b6700 1 --2- 172.21.15.129:0/353821693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c103470 0x7f289c103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.741+0000 7f28a40b6700 1 --2- 172.21.15.129:0/353821693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 0x7f289c104ad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.741+0000 7f28a40b6700 1 -- 172.21.15.129:0/353821693 >> 172.21.15.129:0/353821693 conn(0x7f289c0fea10 msgr2=0x7f289c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:13.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.741+0000 7f28a40b6700 1 -- 172.21.15.129:0/353821693 shutdown_connections 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.741+0000 7f28a40b6700 1 -- 172.21.15.129:0/353821693 wait complete. 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a40b6700 1 Processor -- start 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a40b6700 1 -- start start 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a40b6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f289c103470 0x7f289c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a40b6700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 0x7f289c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a40b6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c105010 0x7f289c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a40b6700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f289c10ef80 con 0x7f289c104670 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a40b6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f289c10ee00 con 0x7f289c105010 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a40b6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f289c10f100 con 0x7f289c103470 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1651700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 0x7f289c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1651700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 0x7f289c199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38556/0 (socket says 172.21.15.129:38556) 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1651700 1 -- 172.21.15.129:0/3400007379 learned_addr learned my addr 172.21.15.129:0/3400007379 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1e52700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f289c103470 0x7f289c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a2653700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c105010 0x7f289c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1651700 1 -- 172.21.15.129:0/3400007379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f289c103470 msgr2=0x7f289c199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1651700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f289c103470 0x7f289c199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1651700 1 -- 172.21.15.129:0/3400007379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c105010 msgr2=0x7f289c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1651700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c105010 0x7f289c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1651700 1 -- 172.21.15.129:0/3400007379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c1a1280 con 0x7f289c104670 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a2653700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c105010 0x7f289c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.742+0000 7f28a1651700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 0x7f289c199b60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2898005910 tx=0x7f2898009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.743+0000 7f2892ffd700 1 -- 172.21.15.129:0/3400007379 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898027070 con 0x7f289c104670 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.743+0000 7f2892ffd700 1 -- 172.21.15.129:0/3400007379 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28980041a0 con 0x7f289c104670 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.743+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c1a1510 con 0x7f289c104670 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.743+0000 7f2892ffd700 1 -- 172.21.15.129:0/3400007379 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898022da0 con 0x7f289c104670 2024-04-26T21:15:13.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.743+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c1a1a40 con 0x7f289c104670 2024-04-26T21:15:13.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.744+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289c06a110 con 0x7f289c104670 2024-04-26T21:15:13.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.745+0000 7f2892ffd700 1 -- 172.21.15.129:0/3400007379 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f289802a070 con 0x7f289c104670 2024-04-26T21:15:13.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.745+0000 7f2892ffd700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28880a3cc0 0x7f28880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.745+0000 7f28a1e52700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28880a3cc0 0x7f28880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.745+0000 7f2892ffd700 1 -- 172.21.15.129:0/3400007379 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f28980d1460 con 0x7f289c104670 2024-04-26T21:15:13.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.746+0000 7f28a1e52700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28880a3cc0 0x7f28880a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f289c1044d0 tx=0x7f288c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:13.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.749+0000 7f2892ffd700 1 -- 172.21.15.129:0/3400007379 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2898031020 con 0x7f289c104670 2024-04-26T21:15:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.949+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f289c02e490 con 0x7f289c104670 2024-04-26T21:15:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.950+0000 7f2892ffd700 1 -- 172.21.15.129:0/3400007379 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f28980a4030 con 0x7f289c104670 2024-04-26T21:15:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28880a3cc0 msgr2=0x7f28880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28880a3cc0 0x7f28880a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f289c1044d0 tx=0x7f288c009190 comp rx=0 tx=0).stop 2024-04-26T21:15:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 msgr2=0x7f289c199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 0x7f289c199b60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2898005910 tx=0x7f2898009510 comp rx=0 tx=0).stop 2024-04-26T21:15:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 shutdown_connections 2024-04-26T21:15:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28880a3cc0 0x7f28880a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c105010 0x7f289c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f289c103470 0x7f289c199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 --2- 172.21.15.129:0/3400007379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f289c104670 0x7f289c199b60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 >> 172.21.15.129:0/3400007379 conn(0x7f289c0fea10 msgr2=0x7f289c107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 shutdown_connections 2024-04-26T21:15:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:13.952+0000 7f28a40b6700 1 -- 172.21.15.129:0/3400007379 wait complete. 2024-04-26T21:15:14.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:14 smithi171 ceph-mon[33478]: purged_snaps scrub starts 2024-04-26T21:15:14.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:14 smithi171 ceph-mon[33478]: purged_snaps scrub ok 2024-04-26T21:15:14.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:14 smithi171 ceph-mon[33478]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:14.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:14 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:14.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:14 smithi171 ceph-mon[33478]: osd.11 [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] boot 2024-04-26T21:15:14.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:14 smithi171 ceph-mon[33478]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T21:15:14.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:14 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:14.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:14 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3400007379' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:15:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:14 smithi150 ceph-mon[32290]: purged_snaps scrub starts 2024-04-26T21:15:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:14 smithi150 ceph-mon[32290]: purged_snaps scrub ok 2024-04-26T21:15:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:14 smithi150 ceph-mon[32290]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:14 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:14 smithi150 ceph-mon[32290]: osd.11 [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] boot 2024-04-26T21:15:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:14 smithi150 ceph-mon[32290]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T21:15:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:14 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:14 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3400007379' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:15:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:14 smithi129 ceph-mon[29237]: purged_snaps scrub starts 2024-04-26T21:15:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:14 smithi129 ceph-mon[29237]: purged_snaps scrub ok 2024-04-26T21:15:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:14 smithi129 ceph-mon[29237]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:15:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:14 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:14 smithi129 ceph-mon[29237]: osd.11 [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] boot 2024-04-26T21:15:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:14 smithi129 ceph-mon[29237]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T21:15:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:14 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:15:14.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:14 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3400007379' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:15:14.779 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714166113,"num_in_osds":12,"osd_in_since":1714166092,"num_remapped_pgs":0} 2024-04-26T21:15:14.780 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:15:15.073 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: Detected new or changed devices on smithi171 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-26T21:15:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:15.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:15.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:15 smithi129 ceph-mon[29237]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:15:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: Detected new or changed devices on smithi171 2024-04-26T21:15:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:15 smithi171 ceph-mon[33478]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:15:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: Detected new or changed devices on smithi171 2024-04-26T21:15:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:15:15.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-26T21:15:15.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:15.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:15.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:15.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:15 smithi150 ceph-mon[32290]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:16.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.780+0000 7fcdf8550700 1 -- 172.21.15.129:0/767197134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102320 msgr2=0x7fcdf01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:16.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.780+0000 7fcdf8550700 1 --2- 172.21.15.129:0/767197134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102320 0x7fcdf01027a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009990 tx=0x7fcdec009ca0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.780+0000 7fcdf8550700 1 -- 172.21.15.129:0/767197134 shutdown_connections 2024-04-26T21:15:16.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.780+0000 7fcdf8550700 1 --2- 172.21.15.129:0/767197134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf0102ce0 0x7fcdf010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.780+0000 7fcdf8550700 1 --2- 172.21.15.129:0/767197134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf0108390 0x7fcdf0108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.780+0000 7fcdf8550700 1 --2- 172.21.15.129:0/767197134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102320 0x7fcdf01027a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.780+0000 7fcdf8550700 1 -- 172.21.15.129:0/767197134 >> 172.21.15.129:0/767197134 conn(0x7fcdf00792d0 msgr2=0x7fcdf00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.780+0000 7fcdf8550700 1 -- 172.21.15.129:0/767197134 shutdown_connections 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.780+0000 7fcdf8550700 1 -- 172.21.15.129:0/767197134 wait complete. 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf8550700 1 Processor -- start 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf8550700 1 -- start start 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf8550700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf0102320 0x7fcdf019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf8550700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102ce0 0x7fcdf019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf8550700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf0108390 0x7fcdf0199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf8550700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf01125b0 con 0x7fcdf0102ce0 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf8550700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0112430 con 0x7fcdf0108390 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf8550700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0112730 con 0x7fcdf0102320 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf5aeb700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102ce0 0x7fcdf019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:16.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf5aeb700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102ce0 0x7fcdf019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38584/0 (socket says 172.21.15.129:38584) 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf5aeb700 1 -- 172.21.15.129:0/2816462466 learned_addr learned my addr 172.21.15.129:0/2816462466 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf6aed700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf0108390 0x7fcdf0199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf62ec700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf0102320 0x7fcdf019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf5aeb700 1 -- 172.21.15.129:0/2816462466 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf0102320 msgr2=0x7fcdf019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf5aeb700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf0102320 0x7fcdf019f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf5aeb700 1 -- 172.21.15.129:0/2816462466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf0108390 msgr2=0x7fcdf0199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf5aeb700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf0108390 0x7fcdf0199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf5aeb700 1 -- 172.21.15.129:0/2816462466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf019a180 con 0x7fcdf0102ce0 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf6aed700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf0108390 0x7fcdf0199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.781+0000 7fcdf5aeb700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102ce0 0x7fcdf019fda0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000c00 tx=0x7fcdec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.782+0000 7fcde77fe700 1 -- 172.21.15.129:0/2816462466 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec027070 con 0x7fcdf0102ce0 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.782+0000 7fcde77fe700 1 -- 172.21.15.129:0/2816462466 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdec0041d0 con 0x7fcdf0102ce0 2024-04-26T21:15:16.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.782+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf019a410 con 0x7fcdf0102ce0 2024-04-26T21:15:16.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.782+0000 7fcde77fe700 1 -- 172.21.15.129:0/2816462466 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00f460 con 0x7fcdf0102ce0 2024-04-26T21:15:16.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.782+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf01a6730 con 0x7fcdf0102ce0 2024-04-26T21:15:16.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.783+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf0103a40 con 0x7fcdf0102ce0 2024-04-26T21:15:16.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.784+0000 7fcde77fe700 1 -- 172.21.15.129:0/2816462466 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fcdec02a070 con 0x7fcdf0102ce0 2024-04-26T21:15:16.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.784+0000 7fcde77fe700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcddc0a3c40 0x7fcddc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:16.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.784+0000 7fcdf62ec700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcddc0a3c40 0x7fcddc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:16.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.784+0000 7fcde77fe700 1 -- 172.21.15.129:0/2816462466 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcdec0d0f40 con 0x7fcdf0102ce0 2024-04-26T21:15:16.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.785+0000 7fcdf62ec700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcddc0a3c40 0x7fcddc0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcdf019ab60 tx=0x7fcde0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:16.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.789+0000 7fcde77fe700 1 -- 172.21.15.129:0/2816462466 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcdec05f060 con 0x7fcdf0102ce0 2024-04-26T21:15:16.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.989+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdf01a6900 con 0x7fcdf0102ce0 2024-04-26T21:15:16.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.990+0000 7fcde77fe700 1 -- 172.21.15.129:0/2816462466 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18511 (secure 0 0 0) 0x7fcdec013070 con 0x7fcdf0102ce0 2024-04-26T21:15:16.992 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:16.992 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":82,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:15:14.397240+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:16.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcddc0a3c40 msgr2=0x7fcddc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:16.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcddc0a3c40 0x7fcddc0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcdf019ab60 tx=0x7fcde0008040 comp rx=0 tx=0).stop 2024-04-26T21:15:16.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102ce0 msgr2=0x7fcdf019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:16.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102ce0 0x7fcdf019fda0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000c00 tx=0x7fcdec0047b0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 shutdown_connections 2024-04-26T21:15:16.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcddc0a3c40 0x7fcddc0a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf0108390 0x7fcdf0199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf0102320 0x7fcdf019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 --2- 172.21.15.129:0/2816462466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdf0102ce0 0x7fcdf019fda0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 >> 172.21.15.129:0/2816462466 conn(0x7fcdf00792d0 msgr2=0x7fcdf0101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:16.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 shutdown_connections 2024-04-26T21:15:16.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:16.992+0000 7fcdf8550700 1 -- 172.21.15.129:0/2816462466 wait complete. 2024-04-26T21:15:17.363 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:17 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2816462466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:15:17.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:17 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2816462466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:15:17.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:17 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2816462466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:15:17.793 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:15:17.794 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T21:15:18.029 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:15:18.300 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:18 smithi129 ceph-mon[29237]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:18.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:18 smithi150 ceph-mon[32290]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:18.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:18 smithi171 ceph-mon[33478]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:19.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.798+0000 7fc77d22c700 1 -- 172.21.15.129:0/397459228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc77806bf00 msgr2=0x7fc77806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.798+0000 7fc77d22c700 1 --2- 172.21.15.129:0/397459228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc77806bf00 0x7fc77806c360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009990 tx=0x7fc76c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.798+0000 7fc77d22c700 1 -- 172.21.15.129:0/397459228 shutdown_connections 2024-04-26T21:15:19.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.798+0000 7fc77d22c700 1 --2- 172.21.15.129:0/397459228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc778105e50 0x7fc7781062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.798+0000 7fc77d22c700 1 --2- 172.21.15.129:0/397459228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc778108340 0x7fc77806b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.798+0000 7fc77d22c700 1 --2- 172.21.15.129:0/397459228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc77806bf00 0x7fc77806c360 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.798+0000 7fc77d22c700 1 -- 172.21.15.129:0/397459228 >> 172.21.15.129:0/397459228 conn(0x7fc778079290 msgr2=0x7fc7780796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:19.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.798+0000 7fc77d22c700 1 -- 172.21.15.129:0/397459228 shutdown_connections 2024-04-26T21:15:19.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.798+0000 7fc77d22c700 1 -- 172.21.15.129:0/397459228 wait complete. 2024-04-26T21:15:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77d22c700 1 Processor -- start 2024-04-26T21:15:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77d22c700 1 -- start start 2024-04-26T21:15:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77d22c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc77806bf00 0x7fc7781a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77d22c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc778105e50 0x7fc7781a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77d22c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc778108340 0x7fc7781a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77d22c700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc778119c00 con 0x7fc778105e50 2024-04-26T21:15:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77d22c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc778119a80 con 0x7fc778108340 2024-04-26T21:15:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77d22c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc778119d80 con 0x7fc77806bf00 2024-04-26T21:15:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77659c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc778105e50 0x7fc7781a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77659c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc778105e50 0x7fc7781a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38604/0 (socket says 172.21.15.129:38604) 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77659c700 1 -- 172.21.15.129:0/1119093519 learned_addr learned my addr 172.21.15.129:0/1119093519 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc77759e700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc778108340 0x7fc7781a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.799+0000 7fc776d9d700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc77806bf00 0x7fc7781a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc77659c700 1 -- 172.21.15.129:0/1119093519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc77806bf00 msgr2=0x7fc7781a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc77659c700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc77806bf00 0x7fc7781a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc77659c700 1 -- 172.21.15.129:0/1119093519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc778108340 msgr2=0x7fc7781a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc77659c700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc778108340 0x7fc7781a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc77659c700 1 -- 172.21.15.129:0/1119093519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7781a5140 con 0x7fc778105e50 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc77659c700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc778105e50 0x7fc7781a0670 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc76c000c00 tx=0x7fc76c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc75ffff700 1 -- 172.21.15.129:0/1119093519 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c027070 con 0x7fc778105e50 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7781a53d0 con 0x7fc778105e50 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc75ffff700 1 -- 172.21.15.129:0/1119093519 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c003fe0 con 0x7fc778105e50 2024-04-26T21:15:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc75ffff700 1 -- 172.21.15.129:0/1119093519 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c022e30 con 0x7fc778105e50 2024-04-26T21:15:19.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.800+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7781accb0 con 0x7fc778105e50 2024-04-26T21:15:19.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.801+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc778102930 con 0x7fc778105e50 2024-04-26T21:15:19.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.802+0000 7fc75ffff700 1 -- 172.21.15.129:0/1119093519 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc76c02a070 con 0x7fc778105e50 2024-04-26T21:15:19.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.803+0000 7fc75ffff700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc7600a3c40 0x7fc7600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.803+0000 7fc776d9d700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc7600a3c40 0x7fc7600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.803+0000 7fc75ffff700 1 -- 172.21.15.129:0/1119093519 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc76c0d0e90 con 0x7fc778105e50 2024-04-26T21:15:19.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.803+0000 7fc776d9d700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc7600a3c40 0x7fc7600a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc768005950 tx=0x7fc76800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:19.806+0000 7fc75ffff700 1 -- 172.21.15.129:0/1119093519 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc76c013070 con 0x7fc778105e50 2024-04-26T21:15:20.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.003+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7781ace80 con 0x7fc778105e50 2024-04-26T21:15:20.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.003+0000 7fc75ffff700 1 -- 172.21.15.129:0/1119093519 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fc76c09dda0 con 0x7fc778105e50 2024-04-26T21:15:20.005 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:15:20.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.005+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc7600a3c40 msgr2=0x7fc7600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.005+0000 7fc77d22c700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc7600a3c40 0x7fc7600a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc768005950 tx=0x7fc76800a300 comp rx=0 tx=0).stop 2024-04-26T21:15:20.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.005+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc778105e50 msgr2=0x7fc7781a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.005+0000 7fc77d22c700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc778105e50 0x7fc7781a0670 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc76c000c00 tx=0x7fc76c0047b0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.006+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 shutdown_connections 2024-04-26T21:15:20.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.006+0000 7fc77d22c700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc7600a3c40 0x7fc7600a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.006+0000 7fc77d22c700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc778108340 0x7fc7781a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.006+0000 7fc77d22c700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc77806bf00 0x7fc7781a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.006+0000 7fc77d22c700 1 --2- 172.21.15.129:0/1119093519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc778105e50 0x7fc7781a0670 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.006+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 >> 172.21.15.129:0/1119093519 conn(0x7fc778079290 msgr2=0x7fc778104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:20.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.006+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 shutdown_connections 2024-04-26T21:15:20.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:20.006+0000 7fc77d22c700 1 -- 172.21.15.129:0/1119093519 wait complete. 2024-04-26T21:15:20.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:20 smithi171 ceph-mon[33478]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:20.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:20 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1119093519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:15:20.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:20 smithi150 ceph-mon[32290]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:20 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1119093519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:15:20.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:20 smithi129 ceph-mon[29237]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:20.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:20 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1119093519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:15:20.750 INFO:tasks.cephadm:Adding mds.a on smithi129 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.d on smithi129 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.g on smithi129 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.j on smithi129 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.b on smithi150 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.e on smithi150 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.h on smithi150 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.k on smithi150 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.c on smithi171 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.f on smithi171 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.i on smithi171 2024-04-26T21:15:20.751 INFO:tasks.cephadm:Adding mds.l on smithi171 2024-04-26T21:15:20.752 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l' 2024-04-26T21:15:20.994 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:15:22.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:22 smithi171 ceph-mon[33478]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:22 smithi150 ceph-mon[32290]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:22.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.679+0000 7fee17f38700 1 -- 172.21.15.171:0/2116863832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102600 msgr2=0x7fee10102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:22.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.679+0000 7fee17f38700 1 --2- 172.21.15.171:0/2116863832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102600 0x7fee10102a80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee00009990 tx=0x7fee00009ca0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.679+0000 7fee17f38700 1 -- 172.21.15.171:0/2116863832 shutdown_connections 2024-04-26T21:15:22.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.679+0000 7fee17f38700 1 --2- 172.21.15.171:0/2116863832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10102fc0 0x7fee1010a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.679+0000 7fee17f38700 1 --2- 172.21.15.171:0/2116863832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee101039a0 0x7fee10103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.679+0000 7fee17f38700 1 --2- 172.21.15.171:0/2116863832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102600 0x7fee10102a80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.679+0000 7fee17f38700 1 -- 172.21.15.171:0/2116863832 >> 172.21.15.171:0/2116863832 conn(0x7fee100fd090 msgr2=0x7fee100ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:22.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 -- 172.21.15.171:0/2116863832 shutdown_connections 2024-04-26T21:15:22.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 -- 172.21.15.171:0/2116863832 wait complete. 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 Processor -- start 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 -- start start 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10102600 0x7fee101994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102fc0 0x7fee10199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee101039a0 0x7fee101a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fee1010de30 con 0x7fee10102600 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee1010dcb0 con 0x7fee101039a0 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee17f38700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fee1010dfb0 con 0x7fee10102fc0 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee154d3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102fc0 0x7fee10199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee154d3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102fc0 0x7fee10199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:41328/0 (socket says 172.21.15.171:41328) 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee154d3700 1 -- 172.21.15.171:0/3193937093 learned_addr learned my addr 172.21.15.171:0/3193937093 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.680+0000 7fee15cd4700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10102600 0x7fee101994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:22.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee164d5700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee101039a0 0x7fee101a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee154d3700 1 -- 172.21.15.171:0/3193937093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee101039a0 msgr2=0x7fee101a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee154d3700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee101039a0 0x7fee101a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee154d3700 1 -- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10102600 msgr2=0x7fee101994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee154d3700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10102600 0x7fee101994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee154d3700 1 -- 172.21.15.171:0/3193937093 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee101a1150 con 0x7fee10102fc0 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee154d3700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102fc0 0x7fee10199a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee000059d0 tx=0x7fee000057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee06ffd700 1 -- 172.21.15.171:0/3193937093 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00027070 con 0x7fee10102fc0 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee101a13e0 con 0x7fee10102fc0 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee06ffd700 1 -- 172.21.15.171:0/3193937093 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee0000fb80 con 0x7fee10102fc0 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee06ffd700 1 -- 172.21.15.171:0/3193937093 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00017650 con 0x7fee10102fc0 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.681+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee101a1890 con 0x7fee10102fc0 2024-04-26T21:15:22.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.682+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1006a110 con 0x7fee10102fc0 2024-04-26T21:15:22.687 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.683+0000 7fee06ffd700 1 -- 172.21.15.171:0/3193937093 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fee0002a070 con 0x7fee10102fc0 2024-04-26T21:15:22.687 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.684+0000 7fee06ffd700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fedfc0a3c40 0x7fedfc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:22.687 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.684+0000 7fee06ffd700 1 -- 172.21.15.171:0/3193937093 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fee000d1120 con 0x7fee10102fc0 2024-04-26T21:15:22.687 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.684+0000 7fee15cd4700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fedfc0a3c40 0x7fedfc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:22.688 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.684+0000 7fee15cd4700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fedfc0a3c40 0x7fedfc0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee1006b9b0 tx=0x7fee0c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:22.690 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.688+0000 7fee06ffd700 1 -- 172.21.15.171:0/3193937093 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fee0005f060 con 0x7fee10102fc0 2024-04-26T21:15:22.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:22 smithi129 ceph-mon[29237]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:22.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.896+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}) -- 0x7fee100ff2f0 con 0x7fedfc0a3c40 2024-04-26T21:15:22.906 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.904+0000 7fee06ffd700 1 -- 172.21.15.171:0/3193937093 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fee100ff2f0 con 0x7fedfc0a3c40 2024-04-26T21:15:22.906 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mds.all update... 2024-04-26T21:15:22.908 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fedfc0a3c40 msgr2=0x7fedfc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:22.908 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fedfc0a3c40 0x7fedfc0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee1006b9b0 tx=0x7fee0c00a3b0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102fc0 msgr2=0x7fee10199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102fc0 0x7fee10199a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee000059d0 tx=0x7fee000057a0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 shutdown_connections 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fedfc0a3c40 0x7fedfc0a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee101039a0 0x7fee101a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10102600 0x7fee101994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 --2- 172.21.15.171:0/3193937093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee10102fc0 0x7fee10199a30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.906+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 >> 172.21.15.171:0/3193937093 conn(0x7fee100fd090 msgr2=0x7fee100feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.907+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 shutdown_connections 2024-04-26T21:15:22.909 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-26T21:15:22.907+0000 7fee17f38700 1 -- 172.21.15.171:0/3193937093 wait complete. 2024-04-26T21:15:23.612 DEBUG:teuthology.orchestra.run.smithi129:mds.a> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T21:15:23.615 DEBUG:teuthology.orchestra.run.smithi129:mds.d> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T21:15:23.661 DEBUG:teuthology.orchestra.run.smithi129:mds.g> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T21:15:23.704 DEBUG:teuthology.orchestra.run.smithi129:mds.j> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T21:15:23.747 DEBUG:teuthology.orchestra.run.smithi150:mds.b> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T21:15:23.749 DEBUG:teuthology.orchestra.run.smithi150:mds.e> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T21:15:23.792 DEBUG:teuthology.orchestra.run.smithi150:mds.h> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T21:15:23.835 DEBUG:teuthology.orchestra.run.smithi150:mds.k> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T21:15:23.878 DEBUG:teuthology.orchestra.run.smithi171:mds.c> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T21:15:23.881 DEBUG:teuthology.orchestra.run.smithi171:mds.f> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T21:15:23.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:15:23.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: Saving service mds.all spec with placement smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-26T21:15:23.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:23.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:23.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:23.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:23.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:23.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:23.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:23.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:23.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:23.923 DEBUG:teuthology.orchestra.run.smithi171:mds.i> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T21:15:23.967 DEBUG:teuthology.orchestra.run.smithi171:mds.l> sudo journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T21:15:24.009 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T21:15:24.010 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.118+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1250580571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94104550 msgr2=0x7fbe9410ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.118+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1250580571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94104550 0x7fbe9410ade0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009ca0 tx=0x7fbe8c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.118+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1250580571 shutdown_connections 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.118+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1250580571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe94103b90 0x7fbe94104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.118+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1250580571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94104550 0x7fbe9410ade0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.118+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1250580571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe941029a0 0x7fbe94102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.118+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1250580571 >> 172.21.15.129:0/1250580571 conn(0x7fbe940fdf30 msgr2=0x7fbe94100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.118+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1250580571 shutdown_connections 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.118+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1250580571 wait complete. 2024-04-26T21:15:24.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.119+0000 7fbe98d9d640 1 Processor -- start 2024-04-26T21:15:24.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.119+0000 7fbe98d9d640 1 -- start start 2024-04-26T21:15:24.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.119+0000 7fbe98d9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe941029a0 0x7fbe94198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.119+0000 7fbe98d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94103b90 0x7fbe941990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.119+0000 7fbe98d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe94104550 0x7fbe941a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.119+0000 7fbe98d9d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410e4b0 con 0x7fbe94103b90 2024-04-26T21:15:24.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.119+0000 7fbe98d9d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410e330 con 0x7fbe941029a0 2024-04-26T21:15:24.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.119+0000 7fbe98d9d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410e630 con 0x7fbe94104550 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe89d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94103b90 0x7fbe941990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe89d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94103b90 0x7fbe941990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45756/0 (socket says 172.21.15.129:45756) 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe89d9b640 1 -- 172.21.15.129:0/1080779168 learned_addr learned my addr 172.21.15.129:0/1080779168 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe92d9d640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe94104550 0x7fbe941a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe9259c640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe941029a0 0x7fbe94198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe89d9b640 1 -- 172.21.15.129:0/1080779168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe94104550 msgr2=0x7fbe941a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe89d9b640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe94104550 0x7fbe941a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe89d9b640 1 -- 172.21.15.129:0/1080779168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe941029a0 msgr2=0x7fbe94198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe89d9b640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe941029a0 0x7fbe94198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe89d9b640 1 -- 172.21.15.129:0/1080779168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe941a07a0 con 0x7fbe94103b90 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe9259c640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe941029a0 0x7fbe94198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe92d9d640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe94104550 0x7fbe941a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe89d9b640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94103b90 0x7fbe941990c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00e9f0 tx=0x7fbe7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:24.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe8bfff640 1 -- 172.21.15.129:0/1080779168 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00cd80 con 0x7fbe94103b90 2024-04-26T21:15:24.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe941a0a90 con 0x7fbe94103b90 2024-04-26T21:15:24.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe8bfff640 1 -- 172.21.15.129:0/1080779168 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe7c0108e0 con 0x7fbe94103b90 2024-04-26T21:15:24.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe8bfff640 1 -- 172.21.15.129:0/1080779168 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c018980 con 0x7fbe94103b90 2024-04-26T21:15:24.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.120+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe941a1030 con 0x7fbe94103b90 2024-04-26T21:15:24.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.121+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe94109d40 con 0x7fbe94103b90 2024-04-26T21:15:24.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.122+0000 7fbe8bfff640 1 -- 172.21.15.129:0/1080779168 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbe7c010430 con 0x7fbe94103b90 2024-04-26T21:15:24.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.123+0000 7fbe8bfff640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe800a3bb0 0x7fbe800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.123+0000 7fbe9259c640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe800a3bb0 0x7fbe800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.123+0000 7fbe8bfff640 1 -- 172.21.15.129:0/1080779168 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbe7c0d07c0 con 0x7fbe94103b90 2024-04-26T21:15:24.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.123+0000 7fbe9259c640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe800a3bb0 0x7fbe800a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005d00 tx=0x7fbe84005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:24.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.127+0000 7fbe8bfff640 1 -- 172.21.15.129:0/1080779168 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbe7c09d640 con 0x7fbe94103b90 2024-04-26T21:15:24.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: Saving service mds.all spec with placement smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:24.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: Saving service mds.all spec with placement smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:24.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:24.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.307+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe9402dc50 con 0x7fbe94103b90 2024-04-26T21:15:24.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.308+0000 7fbe8bfff640 1 -- 172.21.15.129:0/1080779168 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18511 (secure 0 0 0) 0x7fbe7c09cf40 con 0x7fbe94103b90 2024-04-26T21:15:24.309 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:24.309 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":82,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:15:14.397240+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:24.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe800a3bb0 msgr2=0x7fbe800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe800a3bb0 0x7fbe800a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005d00 tx=0x7fbe84005c90 comp rx=0 tx=0).stop 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94103b90 msgr2=0x7fbe941990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94103b90 0x7fbe941990c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00e9f0 tx=0x7fbe7c00eeb0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 shutdown_connections 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe800a3bb0 0x7fbe800a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe94104550 0x7fbe941a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe94103b90 0x7fbe941990c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 --2- 172.21.15.129:0/1080779168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe941029a0 0x7fbe94198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 >> 172.21.15.129:0/1080779168 conn(0x7fbe940fdf30 msgr2=0x7fbe94106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 shutdown_connections 2024-04-26T21:15:24.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.310+0000 7fbe98d9d640 1 -- 172.21.15.129:0/1080779168 wait complete. 2024-04-26T21:15:24.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:15:24.322 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:15:24.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.422+0000 7fc798571640 1 -- 172.21.15.129:0/3721955837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790102a80 msgr2=0x7fc790102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.422+0000 7fc798571640 1 --2- 172.21.15.129:0/3721955837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790102a80 0x7fc790102f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc780009900 tx=0x7fc78002a410 comp rx=0 tx=0).stop 2024-04-26T21:15:24.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.422+0000 7fc798571640 1 -- 172.21.15.129:0/3721955837 shutdown_connections 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.422+0000 7fc798571640 1 --2- 172.21.15.129:0/3721955837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc790103440 0x7fc790109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.422+0000 7fc798571640 1 --2- 172.21.15.129:0/3721955837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790102a80 0x7fc790102f00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.422+0000 7fc798571640 1 --2- 172.21.15.129:0/3721955837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc790101890 0x7fc790101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 -- 172.21.15.129:0/3721955837 >> 172.21.15.129:0/3721955837 conn(0x7fc7900fce20 msgr2=0x7fc7900ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 -- 172.21.15.129:0/3721955837 shutdown_connections 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 -- 172.21.15.129:0/3721955837 wait complete. 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 Processor -- start 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 -- start start 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790101890 0x7fc790197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc790102a80 0x7fc790198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc790103440 0x7fc79019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc79010d3c0 con 0x7fc790101890 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc79010d240 con 0x7fc790103440 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc798571640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc79010d540 con 0x7fc790102a80 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc79630d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790101890 0x7fc790197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc79630d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790101890 0x7fc790197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45766/0 (socket says 172.21.15.129:45766) 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc79630d640 1 -- 172.21.15.129:0/1125951892 learned_addr learned my addr 172.21.15.129:0/1125951892 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc795b0c640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc790102a80 0x7fc790198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.423+0000 7fc796b0e640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc790103440 0x7fc79019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc79630d640 1 -- 172.21.15.129:0/1125951892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc790102a80 msgr2=0x7fc790198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc79630d640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc790102a80 0x7fc790198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc79630d640 1 -- 172.21.15.129:0/1125951892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc790103440 msgr2=0x7fc79019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc79630d640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc790103440 0x7fc79019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc79630d640 1 -- 172.21.15.129:0/1125951892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79019f6e0 con 0x7fc790101890 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc796b0e640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc790103440 0x7fc79019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc79630d640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790101890 0x7fc790197ac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc788009fd0 tx=0x7fc78800c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc7877fe640 1 -- 172.21.15.129:0/1125951892 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788011470 con 0x7fc790101890 2024-04-26T21:15:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79019f9d0 con 0x7fc790101890 2024-04-26T21:15:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc7877fe640 1 -- 172.21.15.129:0/1125951892 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc788011ac0 con 0x7fc790101890 2024-04-26T21:15:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc7877fe640 1 -- 172.21.15.129:0/1125951892 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78801aab0 con 0x7fc790101890 2024-04-26T21:15:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.424+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79019ff40 con 0x7fc790101890 2024-04-26T21:15:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.425+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc790108c50 con 0x7fc790101890 2024-04-26T21:15:24.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.426+0000 7fc7877fe640 1 -- 172.21.15.129:0/1125951892 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc78801ac50 con 0x7fc790101890 2024-04-26T21:15:24.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.427+0000 7fc7877fe640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc77c0a3bb0 0x7fc77c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.427+0000 7fc795b0c640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc77c0a3bb0 0x7fc77c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.427+0000 7fc7877fe640 1 -- 172.21.15.129:0/1125951892 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc7880d04f0 con 0x7fc790101890 2024-04-26T21:15:24.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.427+0000 7fc795b0c640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc77c0a3bb0 0x7fc77c0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc780000c00 tx=0x7fc7800314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:24.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.431+0000 7fc7877fe640 1 -- 172.21.15.129:0/1125951892 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc788016030 con 0x7fc790101890 2024-04-26T21:15:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.608+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc79002dcf0 con 0x7fc790101890 2024-04-26T21:15:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.608+0000 7fc7877fe640 1 -- 172.21.15.129:0/1125951892 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fc78809d370 con 0x7fc790101890 2024-04-26T21:15:24.609 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:15:24.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc77c0a3bb0 msgr2=0x7fc77c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc77c0a3bb0 0x7fc77c0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc780000c00 tx=0x7fc7800314a0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790101890 msgr2=0x7fc790197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790101890 0x7fc790197ac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc788009fd0 tx=0x7fc78800c270 comp rx=0 tx=0).stop 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 shutdown_connections 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc77c0a3bb0 0x7fc77c0a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc790103440 0x7fc79019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc790102a80 0x7fc790198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 --2- 172.21.15.129:0/1125951892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc790101890 0x7fc790197ac0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 >> 172.21.15.129:0/1125951892 conn(0x7fc7900fce20 msgr2=0x7fc790105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 shutdown_connections 2024-04-26T21:15:24.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.610+0000 7fc798571640 1 -- 172.21.15.129:0/1125951892 wait complete. 2024-04-26T21:15:24.623 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:24.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.718+0000 7f466bcc4640 1 -- 172.21.15.129:0/3500337001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664103440 msgr2=0x7f4664109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.718+0000 7f466bcc4640 1 --2- 172.21.15.129:0/3500337001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664103440 0x7f4664109cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f465c009ca0 tx=0x7f465c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.718+0000 7f466bcc4640 1 -- 172.21.15.129:0/3500337001 shutdown_connections 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.718+0000 7f466bcc4640 1 --2- 172.21.15.129:0/3500337001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4664102a80 0x7f4664102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.718+0000 7f466bcc4640 1 --2- 172.21.15.129:0/3500337001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664103440 0x7f4664109cd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.718+0000 7f466bcc4640 1 --2- 172.21.15.129:0/3500337001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664101890 0x7f4664101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.718+0000 7f466bcc4640 1 -- 172.21.15.129:0/3500337001 >> 172.21.15.129:0/3500337001 conn(0x7f46640fce20 msgr2=0x7f46640ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 -- 172.21.15.129:0/3500337001 shutdown_connections 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 -- 172.21.15.129:0/3500337001 wait complete. 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 Processor -- start 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 -- start start 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664101890 0x7f4664198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4664102a80 0x7f4664199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664103440 0x7f46641a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f466410e540 con 0x7f4664101890 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f466410e3c0 con 0x7f4664102a80 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466bcc4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f466410e6c0 con 0x7f4664103440 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f4669a60640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664101890 0x7f4664198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f4669a60640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664101890 0x7f4664198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45788/0 (socket says 172.21.15.129:45788) 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f4669a60640 1 -- 172.21.15.129:0/2978471276 learned_addr learned my addr 172.21.15.129:0/2978471276 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466a261640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664103440 0x7f46641a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.719+0000 7f466925f640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4664102a80 0x7f4664199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f4669a60640 1 -- 172.21.15.129:0/2978471276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664103440 msgr2=0x7f46641a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f4669a60640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664103440 0x7f46641a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f4669a60640 1 -- 172.21.15.129:0/2978471276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4664102a80 msgr2=0x7f4664199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f4669a60640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4664102a80 0x7f4664199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f4669a60640 1 -- 172.21.15.129:0/2978471276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46641a0730 con 0x7f4664101890 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f466925f640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4664102a80 0x7f4664199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:24.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f4669a60640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664101890 0x7f4664198b10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f466000d950 tx=0x7f466000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:24.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f4656ffd640 1 -- 172.21.15.129:0/2978471276 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660017070 con 0x7f4664101890 2024-04-26T21:15:24.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46641a0a20 con 0x7f4664101890 2024-04-26T21:15:24.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f4656ffd640 1 -- 172.21.15.129:0/2978471276 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4660009c30 con 0x7f4664101890 2024-04-26T21:15:24.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f4656ffd640 1 -- 172.21.15.129:0/2978471276 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466000f620 con 0x7f4664101890 2024-04-26T21:15:24.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.720+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641a0f20 con 0x7f4664101890 2024-04-26T21:15:24.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.721+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4664108b50 con 0x7f4664101890 2024-04-26T21:15:24.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.722+0000 7f4656ffd640 1 -- 172.21.15.129:0/2978471276 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f466001a070 con 0x7f4664101890 2024-04-26T21:15:24.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.723+0000 7f4656ffd640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46500a3bb0 0x7f46500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.723+0000 7f466925f640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46500a3bb0 0x7f46500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.723+0000 7f4656ffd640 1 -- 172.21.15.129:0/2978471276 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f46600cf980 con 0x7f4664101890 2024-04-26T21:15:24.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.723+0000 7f466925f640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46500a3bb0 0x7f46500a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f466419a020 tx=0x7f465800a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:24.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.726+0000 7f4656ffd640 1 -- 172.21.15.129:0/2978471276 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4660009de0 con 0x7f4664101890 2024-04-26T21:15:24.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.940+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f466402dad0 con 0x7f4664101890 2024-04-26T21:15:24.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.941+0000 7f4656ffd640 1 -- 172.21.15.129:0/2978471276 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f466009c800 con 0x7f4664101890 2024-04-26T21:15:24.942 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:24.942 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T21:15:24.942 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 1 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.942+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46500a3bb0 msgr2=0x7f46500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.942+0000 7f466bcc4640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46500a3bb0 0x7f46500a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f466419a020 tx=0x7f465800a5c0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664101890 msgr2=0x7f4664198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664101890 0x7f4664198b10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f466000d950 tx=0x7f466000de10 comp rx=0 tx=0).stop 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 shutdown_connections 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f46500a3bb0 0x7f46500a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4664102a80 0x7f4664199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664103440 0x7f46641a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 --2- 172.21.15.129:0/2978471276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4664101890 0x7f4664198b10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 >> 172.21.15.129:0/2978471276 conn(0x7f46640fce20 msgr2=0x7f4664105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:24.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 shutdown_connections 2024-04-26T21:15:24.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:24.943+0000 7f466bcc4640 1 -- 172.21.15.129:0/2978471276 wait complete. 2024-04-26T21:15:24.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:24 smithi171 ceph-mon[33478]: Deploying daemon mds.l on smithi171 2024-04-26T21:15:24.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:24 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1080779168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:15:24.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:24 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1125951892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:15:25.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:24 smithi129 ceph-mon[29237]: Deploying daemon mds.l on smithi171 2024-04-26T21:15:25.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:24 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1080779168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:15:25.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:24 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1125951892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:15:25.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:24 smithi150 ceph-mon[32290]: Deploying daemon mds.l on smithi171 2024-04-26T21:15:25.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:24 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1080779168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:15:25.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:24 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1125951892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:15:26.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:25 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2978471276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:26.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:25 smithi129 ceph-mon[29237]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:26.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:25 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2978471276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:26.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:25 smithi171 ceph-mon[33478]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:26.242 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:15:26 smithi171 systemd[1]: Starting Ceph mds.l for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:15:26.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:25 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2978471276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:26.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:25 smithi150 ceph-mon[32290]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:26.573 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:15:26 smithi171 podman[63528]: 2024-04-26 21:15:26.153674331 +0000 UTC m=+0.018252575 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:15:26.573 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:15:26 smithi171 podman[63528]: 2024-04-26 21:15:26.310526896 +0000 UTC m=+0.175105181 container create ed733b64450ce881beaeedb48bc21b45d9c017c776193eba6e419fce96f6ab82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:15:26.573 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:15:26 smithi171 podman[63528]: 2024-04-26 21:15:26.571357202 +0000 UTC m=+0.435935445 container init ed733b64450ce881beaeedb48bc21b45d9c017c776193eba6e419fce96f6ab82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:15:26.955 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:26.991 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:15:26 smithi171 podman[63528]: 2024-04-26 21:15:26.574062737 +0000 UTC m=+0.438640983 container start ed733b64450ce881beaeedb48bc21b45d9c017c776193eba6e419fce96f6ab82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T21:15:26.991 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:15:26 smithi171 ceph-mds[63547]: -- 172.21.15.171:0/2403082645 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561fdd9dba40 con 0x561fdd9ae800 2024-04-26T21:15:26.992 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:15:26 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-l[63542]: starting mds.l at 2024-04-26T21:15:26.992 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:15:26 smithi171 bash[63528]: ed733b64450ce881beaeedb48bc21b45d9c017c776193eba6e419fce96f6ab82 2024-04-26T21:15:26.992 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:15:26 smithi171 systemd[1]: Started Ceph mds.l for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.066+0000 7f48d4e6d640 1 -- 172.21.15.129:0/2863815869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0101870 msgr2=0x7f48d0101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.066+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/2863815869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0101870 0x7f48d0101c90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009900 tx=0x7f48c802a410 comp rx=0 tx=0).stop 2024-04-26T21:15:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.066+0000 7f48d4e6d640 1 -- 172.21.15.129:0/2863815869 shutdown_connections 2024-04-26T21:15:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.066+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/2863815869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48d0102a60 0x7f48d0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.066+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/2863815869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48d0103420 0x7f48d0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.066+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/2863815869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0101870 0x7f48d0101c90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.066+0000 7f48d4e6d640 1 -- 172.21.15.129:0/2863815869 >> 172.21.15.129:0/2863815869 conn(0x7f48d00fcde0 msgr2=0x7f48d00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 -- 172.21.15.129:0/2863815869 shutdown_connections 2024-04-26T21:15:27.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 -- 172.21.15.129:0/2863815869 wait complete. 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 Processor -- start 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 -- start start 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48d0101870 0x7f48d006a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0102a60 0x7f48d019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48d0103420 0x7f48d01a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48d010d3a0 con 0x7f48d0102a60 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f48d010d220 con 0x7f48d0103420 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.067+0000 7f48d4e6d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f48d010d520 con 0x7f48d0101870 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48cdd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0102a60 0x7f48d019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48cdd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0102a60 0x7f48d019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45796/0 (socket says 172.21.15.129:45796) 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48cdd9b640 1 -- 172.21.15.129:0/196141549 learned_addr learned my addr 172.21.15.129:0/196141549 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48ced9d640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48d0103420 0x7f48d01a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48ce59c640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48d0101870 0x7f48d006a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48cdd9b640 1 -- 172.21.15.129:0/196141549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48d0101870 msgr2=0x7f48d006a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48cdd9b640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48d0101870 0x7f48d006a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48cdd9b640 1 -- 172.21.15.129:0/196141549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48d0103420 msgr2=0x7f48d01a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48cdd9b640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48d0103420 0x7f48d01a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48cdd9b640 1 -- 172.21.15.129:0/196141549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d01a2a60 con 0x7f48d0102a60 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48cdd9b640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0102a60 0x7f48d019fd20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48c400ebc0 tx=0x7f48c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48bf7fe640 1 -- 172.21.15.129:0/196141549 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4010630 con 0x7f48d0102a60 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d01a2cf0 con 0x7f48d0102a60 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48bf7fe640 1 -- 172.21.15.129:0/196141549 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48c4010c80 con 0x7f48d0102a60 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48bf7fe640 1 -- 172.21.15.129:0/196141549 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4018b60 con 0x7f48d0102a60 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.068+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d01a31e0 con 0x7f48d0102a60 2024-04-26T21:15:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.069+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d0108c10 con 0x7f48d0102a60 2024-04-26T21:15:27.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.070+0000 7f48bf7fe640 1 -- 172.21.15.129:0/196141549 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f48c4018d00 con 0x7f48d0102a60 2024-04-26T21:15:27.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.071+0000 7f48bf7fe640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f48b80a3be0 0x7f48b80a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.071+0000 7f48ce59c640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f48b80a3be0 0x7f48b80a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.071+0000 7f48bf7fe640 1 -- 172.21.15.129:0/196141549 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f48c40d04c0 con 0x7f48d0102a60 2024-04-26T21:15:27.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.071+0000 7f48ce59c640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f48b80a3be0 0x7f48b80a60a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009230 tx=0x7f48c80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:27.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.076+0000 7f48bf7fe640 1 -- 172.21.15.129:0/196141549 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f48c409ccf0 con 0x7f48d0102a60 2024-04-26T21:15:27.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.292+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48d002dd20 con 0x7f48d0102a60 2024-04-26T21:15:27.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.293+0000 7f48bf7fe640 1 -- 172.21.15.129:0/196141549 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f48c40a0ff0 con 0x7f48d0102a60 2024-04-26T21:15:27.294 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:27.294 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:15:27.294 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 2 2024-04-26T21:15:27.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.294+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f48b80a3be0 msgr2=0x7f48b80a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.294+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f48b80a3be0 0x7f48b80a60a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009230 tx=0x7f48c80314a0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.294+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0102a60 msgr2=0x7f48d019fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.295+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0102a60 0x7f48d019fd20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48c400ebc0 tx=0x7f48c400c6a0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.295+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 shutdown_connections 2024-04-26T21:15:27.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.295+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f48b80a3be0 0x7f48b80a60a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.295+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48d0103420 0x7f48d01a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.295+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48d0102a60 0x7f48d019fd20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.295+0000 7f48d4e6d640 1 --2- 172.21.15.129:0/196141549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48d0101870 0x7f48d006a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.295+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 >> 172.21.15.129:0/196141549 conn(0x7f48d00fcde0 msgr2=0x7f48d0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:27.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.295+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 shutdown_connections 2024-04-26T21:15:27.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:27.295+0000 7f48d4e6d640 1 -- 172.21.15.129:0/196141549 wait complete. 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.171:6832/1986119815,v1:172.21.15.171:6833/1986119815] up:boot 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: fsmap 1 up:standby 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:27.628 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: Deploying daemon mds.k on smithi150 2024-04-26T21:15:27.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:27.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:27 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/196141549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.171:6832/1986119815,v1:172.21.15.171:6833/1986119815] up:boot 2024-04-26T21:15:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: fsmap 1 up:standby 2024-04-26T21:15:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:15:27.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:27.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:27.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:27.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:27.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:27.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:27.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: Deploying daemon mds.k on smithi150 2024-04-26T21:15:27.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:27.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:27 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/196141549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.171:6832/1986119815,v1:172.21.15.171:6833/1986119815] up:boot 2024-04-26T21:15:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: fsmap 1 up:standby 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: Deploying daemon mds.k on smithi150 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:27.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:27 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/196141549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:29.133 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:15:29 smithi150 systemd[1]: Starting Ceph mds.k for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:15:29.308 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:29.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 -- 172.21.15.129:0/314346475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 msgr2=0x7f1580109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:29.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 --2- 172.21.15.129:0/314346475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 0x7f1580109cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1578009ca0 tx=0x7f157802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 -- 172.21.15.129:0/314346475 shutdown_connections 2024-04-26T21:15:29.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 --2- 172.21.15.129:0/314346475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1580102a70 0x7f1580102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 --2- 172.21.15.129:0/314346475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 0x7f1580109cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 --2- 172.21.15.129:0/314346475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1580101880 0x7f1580101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 -- 172.21.15.129:0/314346475 >> 172.21.15.129:0/314346475 conn(0x7f15800fce10 msgr2=0x7f15800ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 -- 172.21.15.129:0/314346475 shutdown_connections 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 -- 172.21.15.129:0/314346475 wait complete. 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.418+0000 7f1585579640 1 Processor -- start 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1585579640 1 -- start start 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1585579640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1580101880 0x7f1580198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1585579640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1580102a70 0x7f1580199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1585579640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 0x7f15801a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1585579640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f158010e4d0 con 0x7f1580103430 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1585579640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f158010e350 con 0x7f1580102a70 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1585579640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f158010e650 con 0x7f1580101880 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 0x7f15801a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 0x7f15801a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45822/0 (socket says 172.21.15.129:45822) 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157f7fe640 1 -- 172.21.15.129:0/3795983668 learned_addr learned my addr 172.21.15.129:0/3795983668 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157effd640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1580101880 0x7f1580198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157f7fe640 1 -- 172.21.15.129:0/3795983668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1580101880 msgr2=0x7f1580198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157f7fe640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1580101880 0x7f1580198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157f7fe640 1 -- 172.21.15.129:0/3795983668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1580102a70 msgr2=0x7f1580199080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157f7fe640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1580102a70 0x7f1580199080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157f7fe640 1 -- 172.21.15.129:0/3795983668 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15801a0760 con 0x7f1580103430 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f157f7fe640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 0x7f15801a00f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15780097b0 tx=0x7f1578031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1567fff640 1 -- 172.21.15.129:0/3795983668 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578003bf0 con 0x7f1580103430 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1567fff640 1 -- 172.21.15.129:0/3795983668 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1578003d90 con 0x7f1580103430 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15801a09f0 con 0x7f1580103430 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1567fff640 1 -- 172.21.15.129:0/3795983668 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157802fc30 con 0x7f1580103430 2024-04-26T21:15:29.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.419+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15801a0f40 con 0x7f1580103430 2024-04-26T21:15:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.420+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1580108b40 con 0x7f1580103430 2024-04-26T21:15:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.422+0000 7f1567fff640 1 -- 172.21.15.129:0/3795983668 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1578004040 con 0x7f1580103430 2024-04-26T21:15:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.422+0000 7f1567fff640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15680a3bb0 0x7f15680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:29.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.422+0000 7f157effd640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15680a3bb0 0x7f15680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:29.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.422+0000 7f1567fff640 1 -- 172.21.15.129:0/3795983668 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f15780f1880 con 0x7f1580103430 2024-04-26T21:15:29.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.423+0000 7f157effd640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15680a3bb0 0x7f15680a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f15700058c0 tx=0x7f1570005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:29.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.426+0000 7f1567fff640 1 -- 172.21.15.129:0/3795983668 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f15780be700 con 0x7f1580103430 2024-04-26T21:15:29.514 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:15:29 smithi150 podman[62003]: 2024-04-26 21:15:29.143965424 +0000 UTC m=+0.018452726 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:15:29.514 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:15:29 smithi150 podman[62003]: 2024-04-26 21:15:29.243608186 +0000 UTC m=+0.118095518 container create 7d6369958e722e270a69c661c9e79ae5230ab1648c62c8e3219203623f2d6936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:15:29.514 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:15:29 smithi150 podman[62003]: 2024-04-26 21:15:29.513348885 +0000 UTC m=+0.387836175 container init 7d6369958e722e270a69c661c9e79ae5230ab1648c62c8e3219203623f2d6936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T21:15:29.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.642+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f158002dad0 con 0x7f1580103430 2024-04-26T21:15:29.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.643+0000 7f1567fff640 1 -- 172.21.15.129:0/3795983668 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f15780be000 con 0x7f1580103430 2024-04-26T21:15:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:29.644 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:15:29.644 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 2 2024-04-26T21:15:29.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15680a3bb0 msgr2=0x7f15680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:29.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15680a3bb0 0x7f15680a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f15700058c0 tx=0x7f1570005850 comp rx=0 tx=0).stop 2024-04-26T21:15:29.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 msgr2=0x7f15801a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:29.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 0x7f15801a00f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15780097b0 tx=0x7f1578031840 comp rx=0 tx=0).stop 2024-04-26T21:15:29.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 shutdown_connections 2024-04-26T21:15:29.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15680a3bb0 0x7f15680a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1580102a70 0x7f1580199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1580103430 0x7f15801a00f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 --2- 172.21.15.129:0/3795983668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1580101880 0x7f1580198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:29.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 >> 172.21.15.129:0/3795983668 conn(0x7f15800fce10 msgr2=0x7f1580105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:29.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 shutdown_connections 2024-04-26T21:15:29.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:29.645+0000 7f1585579640 1 -- 172.21.15.129:0/3795983668 wait complete. 2024-04-26T21:15:29.883 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:15:29 smithi150 podman[62003]: 2024-04-26 21:15:29.5160595 +0000 UTC m=+0.390546796 container start 7d6369958e722e270a69c661c9e79ae5230ab1648c62c8e3219203623f2d6936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:15:29.883 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:15:29 smithi150 ceph-mds[62023]: -- 172.21.15.150:0/2494564380 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621e8405a40 con 0x5621e83d8800 2024-04-26T21:15:29.883 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:15:29 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k[62019]: starting mds.k at 2024-04-26T21:15:29.883 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:15:29 smithi150 bash[62003]: 7d6369958e722e270a69c661c9e79ae5230ab1648c62c8e3219203623f2d6936 2024-04-26T21:15:29.883 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:15:29 smithi150 systemd[1]: Started Ceph mds.k for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:30.590 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:30 smithi129 ceph-mon[29237]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:30.590 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:30 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3795983668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:30.590 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:30.590 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:30.590 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:30.590 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:30.590 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:30.590 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:30 smithi150 ceph-mon[32290]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:30 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3795983668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:30 smithi171 ceph-mon[33478]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:30 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3795983668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:30.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:31.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:31 smithi150 ceph-mon[32290]: Deploying daemon mds.j on smithi129 2024-04-26T21:15:31.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:31 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.150:6832/1663214837,v1:172.21.15.150:6833/1663214837] up:boot 2024-04-26T21:15:31.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:31 smithi150 ceph-mon[32290]: fsmap 2 up:standby 2024-04-26T21:15:31.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:31 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:15:31.658 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:31.681 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:31 smithi129 ceph-mon[29237]: Deploying daemon mds.j on smithi129 2024-04-26T21:15:31.681 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:31 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.150:6832/1663214837,v1:172.21.15.150:6833/1663214837] up:boot 2024-04-26T21:15:31.681 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:31 smithi129 ceph-mon[29237]: fsmap 2 up:standby 2024-04-26T21:15:31.681 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:31 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:15:31.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:31 smithi171 ceph-mon[33478]: Deploying daemon mds.j on smithi129 2024-04-26T21:15:31.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:31 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.150:6832/1663214837,v1:172.21.15.150:6833/1663214837] up:boot 2024-04-26T21:15:31.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:31 smithi171 ceph-mon[33478]: fsmap 2 up:standby 2024-04-26T21:15:31.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:31 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:15:31.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.813+0000 7f406d789640 1 -- 172.21.15.129:0/3150355448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068108650 msgr2=0x7f406810aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:31.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.813+0000 7f406d789640 1 --2- 172.21.15.129:0/3150355448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068108650 0x7f406810aaa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4060009ca0 tx=0x7f406002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.813+0000 7f406d789640 1 -- 172.21.15.129:0/3150355448 shutdown_connections 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.813+0000 7f406d789640 1 --2- 172.21.15.129:0/3150355448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068108650 0x7f406810aaa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.813+0000 7f406d789640 1 --2- 172.21.15.129:0/3150355448 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4068105d20 0x7f4068108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.813+0000 7f406d789640 1 --2- 172.21.15.129:0/3150355448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40681033f0 0x7f40681057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.813+0000 7f406d789640 1 -- 172.21.15.129:0/3150355448 >> 172.21.15.129:0/3150355448 conn(0x7f40680fcde0 msgr2=0x7f40680ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.813+0000 7f406d789640 1 -- 172.21.15.129:0/3150355448 shutdown_connections 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.813+0000 7f406d789640 1 -- 172.21.15.129:0/3150355448 wait complete. 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f406d789640 1 Processor -- start 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f406d789640 1 -- start start 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f406d789640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40681033f0 0x7f4068198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f406d789640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068105d20 0x7f40681990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f406d789640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4068108650 0x7f40681a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f406d789640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f406810e4c0 con 0x7f4068105d20 2024-04-26T21:15:31.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f406d789640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f406810e340 con 0x7f4068108650 2024-04-26T21:15:31.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f406d789640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f406810e640 con 0x7f40681033f0 2024-04-26T21:15:31.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f405ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068105d20 0x7f40681990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:31.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f405ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068105d20 0x7f40681990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45840/0 (socket says 172.21.15.129:45840) 2024-04-26T21:15:31.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f405ffff640 1 -- 172.21.15.129:0/2631168029 learned_addr learned my addr 172.21.15.129:0/2631168029 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:31.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f40677fe640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4068108650 0x7f40681a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:31.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f4066ffd640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40681033f0 0x7f4068198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:31.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f405ffff640 1 -- 172.21.15.129:0/2631168029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40681033f0 msgr2=0x7f4068198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:31.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f405ffff640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40681033f0 0x7f4068198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f405ffff640 1 -- 172.21.15.129:0/2631168029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4068108650 msgr2=0x7f40681a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:31.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f405ffff640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4068108650 0x7f40681a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.814+0000 7f405ffff640 1 -- 172.21.15.129:0/2631168029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40681a0790 con 0x7f4068105d20 2024-04-26T21:15:31.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.815+0000 7f405ffff640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068105d20 0x7f40681990b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f405000c970 tx=0x7f405000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:31.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.815+0000 7f4064ff9640 1 -- 172.21.15.129:0/2631168029 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050007bb0 con 0x7f4068105d20 2024-04-26T21:15:31.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.815+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40681a0a80 con 0x7f4068105d20 2024-04-26T21:15:31.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.815+0000 7f4064ff9640 1 -- 172.21.15.129:0/2631168029 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4050007d50 con 0x7f4068105d20 2024-04-26T21:15:31.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.815+0000 7f4064ff9640 1 -- 172.21.15.129:0/2631168029 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050018730 con 0x7f4068105d20 2024-04-26T21:15:31.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.815+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40681a0fd0 con 0x7f4068105d20 2024-04-26T21:15:31.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.816+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40680fe8d0 con 0x7f4068105d20 2024-04-26T21:15:31.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.818+0000 7f4064ff9640 1 -- 172.21.15.129:0/2631168029 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f405000f990 con 0x7f4068105d20 2024-04-26T21:15:31.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.818+0000 7f4064ff9640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f40540a3bb0 0x7f40540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:31.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.818+0000 7f4066ffd640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f40540a3bb0 0x7f40540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:31.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.818+0000 7f4064ff9640 1 -- 172.21.15.129:0/2631168029 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f40500d03b0 con 0x7f4068105d20 2024-04-26T21:15:31.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.819+0000 7f4066ffd640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f40540a3bb0 0x7f40540a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4058009b60 tx=0x7f4058005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:31.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:31.822+0000 7f4064ff9640 1 -- 172.21.15.129:0/2631168029 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4050014030 con 0x7f4068105d20 2024-04-26T21:15:32.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.055+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f406802dad0 con 0x7f4068105d20 2024-04-26T21:15:32.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.056+0000 7f4064ff9640 1 -- 172.21.15.129:0/2631168029 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f405009d230 con 0x7f4068105d20 2024-04-26T21:15:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:32.061 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:15:32.061 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 3 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f40540a3bb0 msgr2=0x7f40540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f40540a3bb0 0x7f40540a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4058009b60 tx=0x7f4058005f90 comp rx=0 tx=0).stop 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068105d20 msgr2=0x7f40681990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068105d20 0x7f40681990b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f405000c970 tx=0x7f405000ce30 comp rx=0 tx=0).stop 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 shutdown_connections 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f40540a3bb0 0x7f40540a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4068108650 0x7f40681a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4068105d20 0x7f40681990b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 --2- 172.21.15.129:0/2631168029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40681033f0 0x7f4068198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 >> 172.21.15.129:0/2631168029 conn(0x7f40680fcde0 msgr2=0x7f4068103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 shutdown_connections 2024-04-26T21:15:32.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:32.059+0000 7f406d789640 1 -- 172.21.15.129:0/2631168029 wait complete. 2024-04-26T21:15:32.207 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:15:32 smithi129 podman[69650]: 2024-04-26 21:15:32.0673811 +0000 UTC m=+0.019507355 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:15:32.207 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:15:32 smithi129 podman[69650]: 2024-04-26 21:15:32.207226803 +0000 UTC m=+0.159353047 container create 5187b1d1621778550ddf546935950e1385fcae05fbcc2feb6158c43606540867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-26T21:15:32.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:32 smithi129 ceph-mon[29237]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:32.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:32 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2631168029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:32.470 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:15:32 smithi129 podman[69650]: 2024-04-26 21:15:32.452086197 +0000 UTC m=+0.404212440 container init 5187b1d1621778550ddf546935950e1385fcae05fbcc2feb6158c43606540867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T21:15:32.470 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:15:32 smithi129 podman[69650]: 2024-04-26 21:15:32.454835489 +0000 UTC m=+0.406961739 container start 5187b1d1621778550ddf546935950e1385fcae05fbcc2feb6158c43606540867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T21:15:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:32 smithi150 ceph-mon[32290]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:32 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2631168029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:32.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:32 smithi171 ceph-mon[33478]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:32.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:32 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2631168029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:32.970 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:15:32 smithi129 ceph-mds[69671]: -- 172.21.15.129:0/1483980293 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f3be4e7a40 con 0x55f3be4ba800 2024-04-26T21:15:32.970 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:15:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-j[69667]: starting mds.j at 2024-04-26T21:15:32.970 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:15:32 smithi129 bash[69650]: 5187b1d1621778550ddf546935950e1385fcae05fbcc2feb6158c43606540867 2024-04-26T21:15:32.970 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:15:32 smithi129 systemd[1]: Started Ceph mds.j for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: Deploying daemon mds.d on smithi129 2024-04-26T21:15:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.129:6834/4106276863,v1:172.21.15.129:6835/4106276863] up:boot 2024-04-26T21:15:33.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: fsmap 3 up:standby 2024-04-26T21:15:33.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:33 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:15:33.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:33.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:33.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:33.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:33.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:33.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:33.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: Deploying daemon mds.d on smithi129 2024-04-26T21:15:33.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:33.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.129:6834/4106276863,v1:172.21.15.129:6835/4106276863] up:boot 2024-04-26T21:15:33.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: fsmap 3 up:standby 2024-04-26T21:15:33.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:33 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:15:33.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:33.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:33.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:33.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:33.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:33.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:33.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: Deploying daemon mds.d on smithi129 2024-04-26T21:15:33.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:33.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.129:6834/4106276863,v1:172.21.15.129:6835/4106276863] up:boot 2024-04-26T21:15:33.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: fsmap 3 up:standby 2024-04-26T21:15:33.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:33 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:15:34.073 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:34.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.174+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1634747416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0103430 msgr2=0x7f5dc0109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:34.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.174+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1634747416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0103430 0x7f5dc0109cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009ca0 tx=0x7f5db802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.174+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1634747416 shutdown_connections 2024-04-26T21:15:34.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.174+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1634747416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dc0102a70 0x7f5dc0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.174+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1634747416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0103430 0x7f5dc0109cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.174+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1634747416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc0101880 0x7f5dc0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.174+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1634747416 >> 172.21.15.129:0/1634747416 conn(0x7f5dc00fce30 msgr2=0x7f5dc00ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.174+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1634747416 shutdown_connections 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.174+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1634747416 wait complete. 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dc4cda640 1 Processor -- start 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dc4cda640 1 -- start start 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dc4cda640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0101880 0x7f5dc019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dc4cda640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc0102a70 0x7f5dc019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dc4cda640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dc0103430 0x7f5dc01a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dc4cda640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc010d420 con 0x7f5dc0101880 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dc4cda640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc010d2a0 con 0x7f5dc0103430 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dc4cda640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc010d5a0 con 0x7f5dc0102a70 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dbe59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0101880 0x7f5dc019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dbe59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0101880 0x7f5dc019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48314/0 (socket says 172.21.15.129:48314) 2024-04-26T21:15:34.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.175+0000 7f5dbe59c640 1 -- 172.21.15.129:0/1519048437 learned_addr learned my addr 172.21.15.129:0/1519048437 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dbdd9b640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc0102a70 0x7f5dc019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dbed9d640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dc0103430 0x7f5dc01a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dbe59c640 1 -- 172.21.15.129:0/1519048437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc0102a70 msgr2=0x7f5dc019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dbe59c640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc0102a70 0x7f5dc019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dbe59c640 1 -- 172.21.15.129:0/1519048437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dc0103430 msgr2=0x7f5dc01a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dbe59c640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dc0103430 0x7f5dc01a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dbe59c640 1 -- 172.21.15.129:0/1519048437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc01a29c0 con 0x7f5dc0101880 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dbe59c640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0101880 0x7f5dc019adb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5db000b450 tx=0x7f5db000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5daf7fe640 1 -- 172.21.15.129:0/1519048437 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0015020 con 0x7f5dc0101880 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5daf7fe640 1 -- 172.21.15.129:0/1519048437 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5db0011d40 con 0x7f5dc0101880 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc01a2cb0 con 0x7f5dc0101880 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5daf7fe640 1 -- 172.21.15.129:0/1519048437 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000f340 con 0x7f5dc0101880 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.176+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc01a31b0 con 0x7f5dc0101880 2024-04-26T21:15:34.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.177+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc01a3440 con 0x7f5dc0101880 2024-04-26T21:15:34.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.179+0000 7f5daf7fe640 1 -- 172.21.15.129:0/1519048437 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5db0018020 con 0x7f5dc0101880 2024-04-26T21:15:34.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.179+0000 7f5daf7fe640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5da80a3bb0 0x7f5da80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:34.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.179+0000 7f5dbdd9b640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5da80a3bb0 0x7f5da80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:34.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.179+0000 7f5daf7fe640 1 -- 172.21.15.129:0/1519048437 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5db00d0110 con 0x7f5dc0101880 2024-04-26T21:15:34.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.180+0000 7f5dbdd9b640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5da80a3bb0 0x7f5da80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5db4005e10 tx=0x7f5db4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:34.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.183+0000 7f5daf7fe640 1 -- 172.21.15.129:0/1519048437 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5db001f020 con 0x7f5dc0101880 2024-04-26T21:15:34.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.398+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dc01a3610 con 0x7f5dc0101880 2024-04-26T21:15:34.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.398+0000 7f5daf7fe640 1 -- 172.21.15.129:0/1519048437 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3052 (secure 0 0 0) 0x7f5db000d040 con 0x7f5dc0101880 2024-04-26T21:15:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:34.400 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:15:34.400 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 4 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5da80a3bb0 msgr2=0x7f5da80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5da80a3bb0 0x7f5da80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5db4005e10 tx=0x7f5db4005da0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0101880 msgr2=0x7f5dc019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0101880 0x7f5dc019adb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5db000b450 tx=0x7f5db000b910 comp rx=0 tx=0).stop 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 shutdown_connections 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5da80a3bb0 0x7f5da80a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dc0103430 0x7f5dc01a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc0102a70 0x7f5dc019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 --2- 172.21.15.129:0/1519048437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc0101880 0x7f5dc019adb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 >> 172.21.15.129:0/1519048437 conn(0x7f5dc00fce30 msgr2=0x7f5dc00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 shutdown_connections 2024-04-26T21:15:34.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:34.400+0000 7f5dc4cda640 1 -- 172.21.15.129:0/1519048437 wait complete. 2024-04-26T21:15:34.670 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:34 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1519048437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:34.950 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:34 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1519048437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:34 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1519048437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:35.296 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:15:35 smithi129 podman[69997]: 2024-04-26 21:15:35.273754029 +0000 UTC m=+0.135304589 container create 5891d7cd780f8e2615b0e37b5e2d377ae3a22cb024b8592f1e9805960f085f3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:15:35.547 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:15:35 smithi129 podman[69997]: 2024-04-26 21:15:35.519064012 +0000 UTC m=+0.380614582 container init 5891d7cd780f8e2615b0e37b5e2d377ae3a22cb024b8592f1e9805960f085f3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:15:35.547 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:15:35 smithi129 podman[69997]: 2024-04-26 21:15:35.521843044 +0000 UTC m=+0.383393614 container start 5891d7cd780f8e2615b0e37b5e2d377ae3a22cb024b8592f1e9805960f085f3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T21:15:35.547 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:15:35 smithi129 ceph-mds[70024]: -- 172.21.15.129:0/723069753 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c18ed6fa40 con 0x55c18ed43000 2024-04-26T21:15:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:35 smithi129 ceph-mon[29237]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:35.970 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:15:35 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-d[70015]: starting mds.d at 2024-04-26T21:15:35.970 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:15:35 smithi129 bash[69997]: 5891d7cd780f8e2615b0e37b5e2d377ae3a22cb024b8592f1e9805960f085f3a 2024-04-26T21:15:35.970 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:15:35 smithi129 systemd[1]: Started Ceph mds.d for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:35.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:35 smithi171 ceph-mon[33478]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:36.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:35 smithi150 ceph-mon[32290]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:36.414 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:36.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.515+0000 7fdf1202a640 1 -- 172.21.15.129:0/609101872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103420 msgr2=0x7fdf0c109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.515+0000 7fdf1202a640 1 --2- 172.21.15.129:0/609101872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103420 0x7fdf0c109cb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009ca0 tx=0x7fdf0402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.515+0000 7fdf1202a640 1 -- 172.21.15.129:0/609101872 shutdown_connections 2024-04-26T21:15:36.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.515+0000 7fdf1202a640 1 --2- 172.21.15.129:0/609101872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c102a60 0x7fdf0c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.515+0000 7fdf1202a640 1 --2- 172.21.15.129:0/609101872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103420 0x7fdf0c109cb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.515+0000 7fdf1202a640 1 --2- 172.21.15.129:0/609101872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf0c101870 0x7fdf0c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 -- 172.21.15.129:0/609101872 >> 172.21.15.129:0/609101872 conn(0x7fdf0c0fcde0 msgr2=0x7fdf0c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 -- 172.21.15.129:0/609101872 shutdown_connections 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 -- 172.21.15.129:0/609101872 wait complete. 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 Processor -- start 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 -- start start 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c101870 0x7fdf0c06a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c102a60 0x7fdf0c19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf0c103420 0x7fdf0c19ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10d410 con 0x7fdf0c102a60 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10d290 con 0x7fdf0c101870 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.516+0000 7fdf1202a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10d590 con 0x7fdf0c103420 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c102a60 0x7fdf0c19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c102a60 0x7fdf0c19d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48358/0 (socket says 172.21.15.129:48358) 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0affd640 1 -- 172.21.15.129:0/4193865125 learned_addr learned my addr 172.21.15.129:0/4193865125 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0b7fe640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c101870 0x7fdf0c06a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0affd640 1 -- 172.21.15.129:0/4193865125 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf0c103420 msgr2=0x7fdf0c19ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0bfff640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf0c103420 0x7fdf0c19ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0affd640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf0c103420 0x7fdf0c19ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0affd640 1 -- 172.21.15.129:0/4193865125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c101870 msgr2=0x7fdf0c06a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0affd640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c101870 0x7fdf0c06a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0affd640 1 -- 172.21.15.129:0/4193865125 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c1a07f0 con 0x7fdf0c102a60 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0b7fe640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c101870 0x7fdf0c06a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf0affd640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c102a60 0x7fdf0c19d8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e9f0 tx=0x7fdf0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf08ff9640 1 -- 172.21.15.129:0/4193865125 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000ccc0 con 0x7fdf0c102a60 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c1a0ae0 con 0x7fdf0c102a60 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf08ff9640 1 -- 172.21.15.129:0/4193865125 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf000108e0 con 0x7fdf0c102a60 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf08ff9640 1 -- 172.21.15.129:0/4193865125 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00018880 con 0x7fdf0c102a60 2024-04-26T21:15:36.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.517+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c1a1030 con 0x7fdf0c102a60 2024-04-26T21:15:36.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.518+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c108b90 con 0x7fdf0c102a60 2024-04-26T21:15:36.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.519+0000 7fdf08ff9640 1 -- 172.21.15.129:0/4193865125 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdf00010430 con 0x7fdf0c102a60 2024-04-26T21:15:36.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.520+0000 7fdf08ff9640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdef40a3bb0 0x7fdef40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.520+0000 7fdf0b7fe640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdef40a3bb0 0x7fdef40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.520+0000 7fdf08ff9640 1 -- 172.21.15.129:0/4193865125 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdf000d03b0 con 0x7fdf0c102a60 2024-04-26T21:15:36.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.520+0000 7fdf0b7fe640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdef40a3bb0 0x7fdef40a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c1028c0 tx=0x7fdefc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:36.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.524+0000 7fdf08ff9640 1 -- 172.21.15.129:0/4193865125 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdf0009d230 con 0x7fdf0c102a60 2024-04-26T21:15:36.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.743+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf0c02dcd0 con 0x7fdf0c102a60 2024-04-26T21:15:36.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.744+0000 7fdf08ff9640 1 -- 172.21.15.129:0/4193865125 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7fdf0009cb30 con 0x7fdf0c102a60 2024-04-26T21:15:36.745 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:36.746 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:15:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 5 2024-04-26T21:15:36.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdef40a3bb0 msgr2=0x7fdef40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdef40a3bb0 0x7fdef40a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c1028c0 tx=0x7fdefc009110 comp rx=0 tx=0).stop 2024-04-26T21:15:36.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c102a60 msgr2=0x7fdf0c19d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c102a60 0x7fdf0c19d8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e9f0 tx=0x7fdf0000edf0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 shutdown_connections 2024-04-26T21:15:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdef40a3bb0 0x7fdef40a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf0c103420 0x7fdf0c19ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c102a60 0x7fdf0c19d8f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 --2- 172.21.15.129:0/4193865125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c101870 0x7fdf0c06a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 >> 172.21.15.129:0/4193865125 conn(0x7fdf0c0fcde0 msgr2=0x7fdf0c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 shutdown_connections 2024-04-26T21:15:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:36.746+0000 7fdf1202a640 1 -- 172.21.15.129:0/4193865125 wait complete. 2024-04-26T21:15:36.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:36.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.129:6836/277014387,v1:172.21.15.129:6837/277014387] up:boot 2024-04-26T21:15:36.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: fsmap 4 up:standby 2024-04-26T21:15:36.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:15:36.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:36.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:36.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:36.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:36.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:36.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:36 smithi129 ceph-mon[29237]: Deploying daemon mds.e on smithi150 2024-04-26T21:15:36.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:36.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.129:6836/277014387,v1:172.21.15.129:6837/277014387] up:boot 2024-04-26T21:15:36.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: fsmap 4 up:standby 2024-04-26T21:15:36.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:15:36.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:36.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:36.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:36.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:36.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:36.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:36 smithi171 ceph-mon[33478]: Deploying daemon mds.e on smithi150 2024-04-26T21:15:37.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:37.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.129:6836/277014387,v1:172.21.15.129:6837/277014387] up:boot 2024-04-26T21:15:37.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: fsmap 4 up:standby 2024-04-26T21:15:37.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:15:37.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:37.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:37.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:37.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:37.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:37.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:36 smithi150 ceph-mon[32290]: Deploying daemon mds.e on smithi150 2024-04-26T21:15:37.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:37 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/4193865125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:37.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:37 smithi129 ceph-mon[29237]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:37.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:37 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/4193865125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:37.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:37 smithi171 ceph-mon[33478]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:37.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:37 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/4193865125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:37.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:37 smithi150 ceph-mon[32290]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:38.633 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 26 21:15:38 smithi150 systemd[1]: Started Ceph mds.e for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:38.759 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:38.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.863+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1636424732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c101890 msgr2=0x7f9f6c101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.863+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1636424732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c101890 0x7f9f6c101cb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009900 tx=0x7f9f6802a410 comp rx=0 tx=0).stop 2024-04-26T21:15:38.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.863+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1636424732 shutdown_connections 2024-04-26T21:15:38.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.863+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1636424732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f6c102a80 0x7f9f6c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.863+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1636424732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f6c103440 0x7f9f6c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.863+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1636424732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c101890 0x7f9f6c101cb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.863+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1636424732 >> 172.21.15.129:0/1636424732 conn(0x7f9f6c0fce20 msgr2=0x7f9f6c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:38.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.863+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1636424732 shutdown_connections 2024-04-26T21:15:38.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.863+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1636424732 wait complete. 2024-04-26T21:15:38.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f73e6d640 1 Processor -- start 2024-04-26T21:15:38.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f73e6d640 1 -- start start 2024-04-26T21:15:38.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f73e6d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f6c101890 0x7f9f6c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f73e6d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f6c102a80 0x7f9f6c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f73e6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c103440 0x7f9f6c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f73e6d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c10d3c0 con 0x7f9f6c103440 2024-04-26T21:15:38.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f73e6d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c10d240 con 0x7f9f6c101890 2024-04-26T21:15:38.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f73e6d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c10d540 con 0x7f9f6c102a80 2024-04-26T21:15:38.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f7240a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c103440 0x7f9f6c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f7240a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c103440 0x7f9f6c1a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48376/0 (socket says 172.21.15.129:48376) 2024-04-26T21:15:38.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f7240a640 1 -- 172.21.15.129:0/1297004834 learned_addr learned my addr 172.21.15.129:0/1297004834 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f71408640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f6c102a80 0x7f9f6c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f71c09640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f6c101890 0x7f9f6c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f7240a640 1 -- 172.21.15.129:0/1297004834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f6c102a80 msgr2=0x7f9f6c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f7240a640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f6c102a80 0x7f9f6c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f7240a640 1 -- 172.21.15.129:0/1297004834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f6c101890 msgr2=0x7f9f6c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f7240a640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f6c101890 0x7f9f6c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f7240a640 1 -- 172.21.15.129:0/1297004834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c1a07a0 con 0x7f9f6c103440 2024-04-26T21:15:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f71c09640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f6c101890 0x7f9f6c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.864+0000 7f9f7240a640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c103440 0x7f9f6c1a0130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400eb10 tx=0x7f9f6400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.865+0000 7f9f62ffd640 1 -- 172.21.15.129:0/1297004834 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6400ce20 con 0x7f9f6c103440 2024-04-26T21:15:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.865+0000 7f9f62ffd640 1 -- 172.21.15.129:0/1297004834 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f64005020 con 0x7f9f6c103440 2024-04-26T21:15:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.865+0000 7f9f62ffd640 1 -- 172.21.15.129:0/1297004834 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64010690 con 0x7f9f6c103440 2024-04-26T21:15:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.865+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6c1a0a90 con 0x7f9f6c103440 2024-04-26T21:15:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.865+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6c1a1030 con 0x7f9f6c103440 2024-04-26T21:15:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.866+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f6c1a1260 con 0x7f9f6c103440 2024-04-26T21:15:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.867+0000 7f9f62ffd640 1 -- 172.21.15.129:0/1297004834 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9f64005510 con 0x7f9f6c103440 2024-04-26T21:15:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.867+0000 7f9f62ffd640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f580a3c30 0x7f9f580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.867+0000 7f9f71c09640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f580a3c30 0x7f9f580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.867+0000 7f9f62ffd640 1 -- 172.21.15.129:0/1297004834 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9f640d0640 con 0x7f9f6c103440 2024-04-26T21:15:38.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.868+0000 7f9f71c09640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f580a3c30 0x7f9f580a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f68000c00 tx=0x7f9f680314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:38.872+0000 7f9f62ffd640 1 -- 172.21.15.129:0/1297004834 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9f6409d4c0 con 0x7f9f6c103440 2024-04-26T21:15:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.085+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f6c1a1430 con 0x7f9f6c103440 2024-04-26T21:15:39.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.086+0000 7f9f62ffd640 1 -- 172.21.15.129:0/1297004834 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f9f6409cdc0 con 0x7f9f6c103440 2024-04-26T21:15:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:39.088 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:15:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 6 2024-04-26T21:15:39.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f580a3c30 msgr2=0x7f9f580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f580a3c30 0x7f9f580a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f68000c00 tx=0x7f9f680314a0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c103440 msgr2=0x7f9f6c1a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c103440 0x7f9f6c1a0130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400eb10 tx=0x7f9f6400ef10 comp rx=0 tx=0).stop 2024-04-26T21:15:39.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 shutdown_connections 2024-04-26T21:15:39.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9f580a3c30 0x7f9f580a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6c103440 0x7f9f6c1a0130 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f6c102a80 0x7f9f6c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 --2- 172.21.15.129:0/1297004834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f6c101890 0x7f9f6c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 >> 172.21.15.129:0/1297004834 conn(0x7f9f6c0fce20 msgr2=0x7f9f6c06af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:39.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 shutdown_connections 2024-04-26T21:15:39.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:39.088+0000 7f9f73e6d640 1 -- 172.21.15.129:0/1297004834 wait complete. 2024-04-26T21:15:39.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:39.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:39.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:39.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:39.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:39.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:39.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: Deploying daemon mds.f on smithi171 2024-04-26T21:15:39.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.150:6834/2260848843,v1:172.21.15.150:6835/2260848843] up:boot 2024-04-26T21:15:39.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: fsmap 5 up:standby 2024-04-26T21:15:39.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:15:39.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1297004834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:39.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:39 smithi171 ceph-mon[33478]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:39.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:39.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:39.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:39.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:39.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:39.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:39.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: Deploying daemon mds.f on smithi171 2024-04-26T21:15:39.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.150:6834/2260848843,v1:172.21.15.150:6835/2260848843] up:boot 2024-04-26T21:15:39.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: fsmap 5 up:standby 2024-04-26T21:15:39.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:15:39.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1297004834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:39.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:39 smithi129 ceph-mon[29237]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:40.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:40.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:40.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:40.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:40.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:40.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:40.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: Deploying daemon mds.f on smithi171 2024-04-26T21:15:40.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.150:6834/2260848843,v1:172.21.15.150:6835/2260848843] up:boot 2024-04-26T21:15:40.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: fsmap 5 up:standby 2024-04-26T21:15:40.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:15:40.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1297004834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:40.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:39 smithi150 ceph-mon[32290]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:41.101 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:41.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.209+0000 7ff24aa7e640 1 -- 172.21.15.129:0/954975391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 msgr2=0x7ff244105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.209+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/954975391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 0x7ff244105860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff234009900 tx=0x7ff23402a410 comp rx=0 tx=0).stop 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.209+0000 7ff24aa7e640 1 -- 172.21.15.129:0/954975391 shutdown_connections 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.209+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/954975391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff244105da0 0x7ff244108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.209+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/954975391 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2441086d0 0x7ff24410ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.209+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/954975391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 0x7ff244105860 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.209+0000 7ff24aa7e640 1 -- 172.21.15.129:0/954975391 >> 172.21.15.129:0/954975391 conn(0x7ff2440fce80 msgr2=0x7ff2440ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.209+0000 7ff24aa7e640 1 -- 172.21.15.129:0/954975391 shutdown_connections 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.209+0000 7ff24aa7e640 1 -- 172.21.15.129:0/954975391 wait complete. 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24aa7e640 1 Processor -- start 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24aa7e640 1 -- start start 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24aa7e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 0x7ff244198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24aa7e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff244105da0 0x7ff244199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24aa7e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2441086d0 0x7ff2441a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24aa7e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff24410e4b0 con 0x7ff244103470 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24aa7e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff24410e330 con 0x7ff2441086d0 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24aa7e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff24410e630 con 0x7ff244105da0 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24881a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 0x7ff244198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24881a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 0x7ff244198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48394/0 (socket says 172.21.15.129:48394) 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff24881a640 1 -- 172.21.15.129:0/3571421449 learned_addr learned my addr 172.21.15.129:0/3571421449 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.210+0000 7ff243fff640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff244105da0 0x7ff244199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff24881a640 1 -- 172.21.15.129:0/3571421449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff244105da0 msgr2=0x7ff244199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff24881a640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff244105da0 0x7ff244199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff24881a640 1 -- 172.21.15.129:0/3571421449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2441086d0 msgr2=0x7ff2441a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff24881a640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2441086d0 0x7ff2441a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff24881a640 1 -- 172.21.15.129:0/3571421449 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2441a0750 con 0x7ff244103470 2024-04-26T21:15:41.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff24881a640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 0x7ff244198b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff234005940 tx=0x7ff234004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff241ffb640 1 -- 172.21.15.129:0/3571421449 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23403f030 con 0x7ff244103470 2024-04-26T21:15:41.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff241ffb640 1 -- 172.21.15.129:0/3571421449 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff234003cb0 con 0x7ff244103470 2024-04-26T21:15:41.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2441a09e0 con 0x7ff244103470 2024-04-26T21:15:41.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff241ffb640 1 -- 172.21.15.129:0/3571421449 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234035070 con 0x7ff244103470 2024-04-26T21:15:41.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.211+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2441a0f20 con 0x7ff244103470 2024-04-26T21:15:41.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.212+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2440fe970 con 0x7ff244103470 2024-04-26T21:15:41.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.213+0000 7ff241ffb640 1 -- 172.21.15.129:0/3571421449 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff234042030 con 0x7ff244103470 2024-04-26T21:15:41.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.214+0000 7ff241ffb640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff22c0a3c30 0x7ff22c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.214+0000 7ff243fff640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff22c0a3c30 0x7ff22c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.214+0000 7ff241ffb640 1 -- 172.21.15.129:0/3571421449 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff2340f0490 con 0x7ff244103470 2024-04-26T21:15:41.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.214+0000 7ff243fff640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff22c0a3c30 0x7ff22c0a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff24419a040 tx=0x7ff23800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.218+0000 7ff241ffb640 1 -- 172.21.15.129:0/3571421449 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2340bd310 con 0x7ff244103470 2024-04-26T21:15:41.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.441+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff24402dcf0 con 0x7ff244103470 2024-04-26T21:15:41.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.441+0000 7ff241ffb640 1 -- 172.21.15.129:0/3571421449 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7ff234003e60 con 0x7ff244103470 2024-04-26T21:15:41.443 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:41.443 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:15:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 6 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff22c0a3c30 msgr2=0x7ff22c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff22c0a3c30 0x7ff22c0a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff24419a040 tx=0x7ff23800a2d0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 msgr2=0x7ff244198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 0x7ff244198b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff234005940 tx=0x7ff234004430 comp rx=0 tx=0).stop 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 shutdown_connections 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff22c0a3c30 0x7ff22c0a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2441086d0 0x7ff2441a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff244105da0 0x7ff244199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 --2- 172.21.15.129:0/3571421449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff244103470 0x7ff244198b30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.443+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 >> 172.21.15.129:0/3571421449 conn(0x7ff2440fce80 msgr2=0x7ff244103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:41.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.444+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 shutdown_connections 2024-04-26T21:15:41.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:41.444+0000 7ff24aa7e640 1 -- 172.21.15.129:0/3571421449 wait complete. 2024-04-26T21:15:41.741 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:15:41 smithi171 systemd[1]: Started Ceph mds.f for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:42.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:42 smithi171 ceph-mon[33478]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:42.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:42 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3571421449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:42.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:42.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:42.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:42.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:42.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:42.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:42 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:42 smithi150 ceph-mon[32290]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:42 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3571421449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:42 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:42 smithi129 ceph-mon[29237]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:42 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3571421449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:42.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:42.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:42.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:42.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:42.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:42 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:43.456 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:43.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:43 smithi171 ceph-mon[33478]: Deploying daemon mds.c on smithi171 2024-04-26T21:15:43.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:43 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.171:6834/3940507923,v1:172.21.15.171:6835/3940507923] up:boot 2024-04-26T21:15:43.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:43 smithi171 ceph-mon[33478]: fsmap 6 up:standby 2024-04-26T21:15:43.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:43 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:15:43.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.566+0000 7fc2266a2640 1 -- 172.21.15.129:0/3364947614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220105d40 msgr2=0x7fc220108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.566+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3364947614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220105d40 0x7fc220108130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc214009900 tx=0x7fc21402a410 comp rx=0 tx=0).stop 2024-04-26T21:15:43.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.566+0000 7fc2266a2640 1 -- 172.21.15.129:0/3364947614 shutdown_connections 2024-04-26T21:15:43.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.566+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3364947614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc220108670 0x7fc22010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.566+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3364947614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220105d40 0x7fc220108130 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.566+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3364947614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220103410 0x7fc220105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.566+0000 7fc2266a2640 1 -- 172.21.15.129:0/3364947614 >> 172.21.15.129:0/3364947614 conn(0x7fc2200fce20 msgr2=0x7fc2200ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.566+0000 7fc2266a2640 1 -- 172.21.15.129:0/3364947614 shutdown_connections 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.566+0000 7fc2266a2640 1 -- 172.21.15.129:0/3364947614 wait complete. 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc2266a2640 1 Processor -- start 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc2266a2640 1 -- start start 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc2266a2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc220103410 0x7fc220198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc2266a2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220105d40 0x7fc2201991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc2266a2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220108670 0x7fc2201a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc2266a2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc22010e540 con 0x7fc220108670 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc2266a2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc22010e3c0 con 0x7fc220105d40 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc2266a2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc22010e6c0 con 0x7fc220103410 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc224c3f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220108670 0x7fc2201a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc224c3f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220108670 0x7fc2201a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34200/0 (socket says 172.21.15.129:34200) 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc224c3f640 1 -- 172.21.15.129:0/3772357922 learned_addr learned my addr 172.21.15.129:0/3772357922 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.567+0000 7fc21ffff640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc220103410 0x7fc220198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc21f7fe640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220105d40 0x7fc2201991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc224c3f640 1 -- 172.21.15.129:0/3772357922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc220103410 msgr2=0x7fc220198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc224c3f640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc220103410 0x7fc220198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc224c3f640 1 -- 172.21.15.129:0/3772357922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220105d40 msgr2=0x7fc2201991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc224c3f640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220105d40 0x7fc2201991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc224c3f640 1 -- 172.21.15.129:0/3772357922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2201a08f0 con 0x7fc220108670 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc21ffff640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc220103410 0x7fc220198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc21f7fe640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220105d40 0x7fc2201991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc224c3f640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220108670 0x7fc2201a0250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc21800eb10 tx=0x7fc21800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc21d7fa640 1 -- 172.21.15.129:0/3772357922 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21800ce20 con 0x7fc220108670 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc21d7fa640 1 -- 172.21.15.129:0/3772357922 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc218005020 con 0x7fc220108670 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2201a0be0 con 0x7fc220108670 2024-04-26T21:15:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc21d7fa640 1 -- 172.21.15.129:0/3772357922 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218010690 con 0x7fc220108670 2024-04-26T21:15:43.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.568+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2201a11b0 con 0x7fc220108670 2024-04-26T21:15:43.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.569+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2200fe910 con 0x7fc220108670 2024-04-26T21:15:43.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.570+0000 7fc21d7fa640 1 -- 172.21.15.129:0/3772357922 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc218005510 con 0x7fc220108670 2024-04-26T21:15:43.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.571+0000 7fc21d7fa640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc2080a3bb0 0x7fc2080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.571+0000 7fc21ffff640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc2080a3bb0 0x7fc2080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.571+0000 7fc21d7fa640 1 -- 172.21.15.129:0/3772357922 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc2180d0a10 con 0x7fc220108670 2024-04-26T21:15:43.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.571+0000 7fc21ffff640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc2080a3bb0 0x7fc2080a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc220199b80 tx=0x7fc210009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.574+0000 7fc21d7fa640 1 -- 172.21.15.129:0/3772357922 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc21809d890 con 0x7fc220108670 2024-04-26T21:15:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:43 smithi150 ceph-mon[32290]: Deploying daemon mds.c on smithi171 2024-04-26T21:15:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:43 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.171:6834/3940507923,v1:172.21.15.171:6835/3940507923] up:boot 2024-04-26T21:15:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:43 smithi150 ceph-mon[32290]: fsmap 6 up:standby 2024-04-26T21:15:43.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:43 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:15:43.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:43 smithi129 ceph-mon[29237]: Deploying daemon mds.c on smithi171 2024-04-26T21:15:43.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:43 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.171:6834/3940507923,v1:172.21.15.171:6835/3940507923] up:boot 2024-04-26T21:15:43.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:43 smithi129 ceph-mon[29237]: fsmap 6 up:standby 2024-04-26T21:15:43.723 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:43 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:15:43.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.790+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc22002dd20 con 0x7fc220108670 2024-04-26T21:15:43.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.791+0000 7fc21d7fa640 1 -- 172.21.15.129:0/3772357922 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7fc21809d190 con 0x7fc220108670 2024-04-26T21:15:43.792 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:43.792 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:15:43.793 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 7 2024-04-26T21:15:43.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc2080a3bb0 msgr2=0x7fc2080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc2080a3bb0 0x7fc2080a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc220199b80 tx=0x7fc210009110 comp rx=0 tx=0).stop 2024-04-26T21:15:43.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220108670 msgr2=0x7fc2201a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220108670 0x7fc2201a0250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc21800eb10 tx=0x7fc21800ef10 comp rx=0 tx=0).stop 2024-04-26T21:15:43.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 shutdown_connections 2024-04-26T21:15:43.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc2080a3bb0 0x7fc2080a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220105d40 0x7fc2201991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc220108670 0x7fc2201a0250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 --2- 172.21.15.129:0/3772357922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc220103410 0x7fc220198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 >> 172.21.15.129:0/3772357922 conn(0x7fc2200fce20 msgr2=0x7fc220103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:43.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 shutdown_connections 2024-04-26T21:15:43.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:43.793+0000 7fc2266a2640 1 -- 172.21.15.129:0/3772357922 wait complete. 2024-04-26T21:15:43.912 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:15:43 smithi171 systemd[1]: Starting Ceph mds.c for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:15:44.241 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:15:43 smithi171 podman[64150]: 2024-04-26 21:15:43.814642676 +0000 UTC m=+0.018341462 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:15:44.241 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:15:43 smithi171 podman[64150]: 2024-04-26 21:15:43.963181323 +0000 UTC m=+0.166880081 container create c058ac99f18fa4e342536a68d6e86bc8276053d211c02cee6e20cb3699e77774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:15:44.241 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:15:44 smithi171 podman[64150]: 2024-04-26 21:15:44.240675797 +0000 UTC m=+0.444374564 container init c058ac99f18fa4e342536a68d6e86bc8276053d211c02cee6e20cb3699e77774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T21:15:44.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:44 smithi171 ceph-mon[33478]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:44.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:44 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3772357922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:44.492 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:15:44 smithi171 podman[64150]: 2024-04-26 21:15:44.243245708 +0000 UTC m=+0.446944467 container start c058ac99f18fa4e342536a68d6e86bc8276053d211c02cee6e20cb3699e77774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS) 2024-04-26T21:15:44.492 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:15:44 smithi171 ceph-mds[64169]: -- 172.21.15.171:0/1268577181 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df242d7a40 con 0x55df242ab000 2024-04-26T21:15:44.492 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:15:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-c[64165]: starting mds.c at 2024-04-26T21:15:44.492 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:15:44 smithi171 bash[64150]: c058ac99f18fa4e342536a68d6e86bc8276053d211c02cee6e20cb3699e77774 2024-04-26T21:15:44.492 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:15:44 smithi171 systemd[1]: Started Ceph mds.c for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:44.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:44 smithi150 ceph-mon[32290]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:44.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:44 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3772357922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:44.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:44 smithi129 ceph-mon[29237]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:44.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:44 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3772357922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:45.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:45 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.171:6836/2148122468,v1:172.21.15.171:6837/2148122468] up:boot 2024-04-26T21:15:45.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:45 smithi150 ceph-mon[32290]: fsmap 7 up:standby 2024-04-26T21:15:45.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:15:45.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:45.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:45.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:45.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:45.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:45.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:45 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:45.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:45 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.171:6836/2148122468,v1:172.21.15.171:6837/2148122468] up:boot 2024-04-26T21:15:45.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:45 smithi129 ceph-mon[29237]: fsmap 7 up:standby 2024-04-26T21:15:45.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:15:45.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:45.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:45.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:45.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:45.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:45.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:45 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:45.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:45 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.171:6836/2148122468,v1:172.21.15.171:6837/2148122468] up:boot 2024-04-26T21:15:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:45 smithi171 ceph-mon[33478]: fsmap 7 up:standby 2024-04-26T21:15:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:15:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:45.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:45 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:45.808 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:45.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.909+0000 7fd86f12b640 1 -- 172.21.15.129:0/4247748462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868103410 msgr2=0x7fd868105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:45.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.909+0000 7fd86f12b640 1 --2- 172.21.15.129:0/4247748462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868103410 0x7fd868105800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd858009900 tx=0x7fd85802a410 comp rx=0 tx=0).stop 2024-04-26T21:15:45.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.909+0000 7fd86f12b640 1 -- 172.21.15.129:0/4247748462 shutdown_connections 2024-04-26T21:15:45.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.909+0000 7fd86f12b640 1 --2- 172.21.15.129:0/4247748462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd868105d40 0x7fd868108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.909+0000 7fd86f12b640 1 --2- 172.21.15.129:0/4247748462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd868108670 0x7fd86810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.909+0000 7fd86f12b640 1 --2- 172.21.15.129:0/4247748462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868103410 0x7fd868105800 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.909+0000 7fd86f12b640 1 -- 172.21.15.129:0/4247748462 >> 172.21.15.129:0/4247748462 conn(0x7fd8680fce20 msgr2=0x7fd8680ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.909+0000 7fd86f12b640 1 -- 172.21.15.129:0/4247748462 shutdown_connections 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.909+0000 7fd86f12b640 1 -- 172.21.15.129:0/4247748462 wait complete. 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86f12b640 1 Processor -- start 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86f12b640 1 -- start start 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86f12b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd868103410 0x7fd868102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86f12b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd868105d40 0x7fd8681013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86f12b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868108670 0x7fd868101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86f12b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd86810e580 con 0x7fd868108670 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86f12b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd86810e400 con 0x7fd868103410 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86f12b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd86810e700 con 0x7fd868105d40 2024-04-26T21:15:45.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86d6c8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868108670 0x7fd868101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86d6c8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868108670 0x7fd868101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34220/0 (socket says 172.21.15.129:34220) 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86d6c8640 1 -- 172.21.15.129:0/1978494542 learned_addr learned my addr 172.21.15.129:0/1978494542 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd867fff640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd868105d40 0x7fd8681013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.910+0000 7fd86cec7640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd868103410 0x7fd868102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd86d6c8640 1 -- 172.21.15.129:0/1978494542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd868105d40 msgr2=0x7fd8681013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd86d6c8640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd868105d40 0x7fd8681013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd86d6c8640 1 -- 172.21.15.129:0/1978494542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd868103410 msgr2=0x7fd868102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd86d6c8640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd868103410 0x7fd868102d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd86d6c8640 1 -- 172.21.15.129:0/1978494542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd868102190 con 0x7fd868108670 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd86cec7640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd868103410 0x7fd868102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd86d6c8640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868108670 0x7fd868101900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd86000b970 tx=0x7fd86000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd865ffb640 1 -- 172.21.15.129:0/1978494542 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860019070 con 0x7fd868108670 2024-04-26T21:15:45.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd865ffb640 1 -- 172.21.15.129:0/1978494542 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd860014440 con 0x7fd868108670 2024-04-26T21:15:45.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8681a2d70 con 0x7fd868108670 2024-04-26T21:15:45.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd865ffb640 1 -- 172.21.15.129:0/1978494542 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8600075d0 con 0x7fd868108670 2024-04-26T21:15:45.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.911+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8681a3280 con 0x7fd868108670 2024-04-26T21:15:45.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.912+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd868069ba0 con 0x7fd868108670 2024-04-26T21:15:45.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.913+0000 7fd865ffb640 1 -- 172.21.15.129:0/1978494542 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd86001c070 con 0x7fd868108670 2024-04-26T21:15:45.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.913+0000 7fd865ffb640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd8500a3c30 0x7fd8500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.914+0000 7fd86cec7640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd8500a3c30 0x7fd8500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.914+0000 7fd865ffb640 1 -- 172.21.15.129:0/1978494542 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd8600cf530 con 0x7fd868108670 2024-04-26T21:15:45.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.914+0000 7fd86cec7640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd8500a3c30 0x7fd8500a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd85802a930 tx=0x7fd858005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:45.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:45.918+0000 7fd865ffb640 1 -- 172.21.15.129:0/1978494542 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd86009c3b0 con 0x7fd868108670 2024-04-26T21:15:46.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.133+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd86802dad0 con 0x7fd868108670 2024-04-26T21:15:46.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.134+0000 7fd865ffb640 1 -- 172.21.15.129:0/1978494542 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7fd86009bcb0 con 0x7fd868108670 2024-04-26T21:15:46.135 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:46.135 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T21:15:46.135 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 8 2024-04-26T21:15:46.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd8500a3c30 msgr2=0x7fd8500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:46.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd8500a3c30 0x7fd8500a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd85802a930 tx=0x7fd858005fd0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868108670 msgr2=0x7fd868101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:46.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868108670 0x7fd868101900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd86000b970 tx=0x7fd86000be30 comp rx=0 tx=0).stop 2024-04-26T21:15:46.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 shutdown_connections 2024-04-26T21:15:46.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd8500a3c30 0x7fd8500a60f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd868108670 0x7fd868101900 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd868105d40 0x7fd8681013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 --2- 172.21.15.129:0/1978494542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd868103410 0x7fd868102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 >> 172.21.15.129:0/1978494542 conn(0x7fd8680fce20 msgr2=0x7fd868103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:46.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 shutdown_connections 2024-04-26T21:15:46.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:46.136+0000 7fd86f12b640 1 -- 172.21.15.129:0/1978494542 wait complete. 2024-04-26T21:15:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:46 smithi129 ceph-mon[29237]: Deploying daemon mds.h on smithi150 2024-04-26T21:15:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:46 smithi129 ceph-mon[29237]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:46.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:46 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1978494542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:46.542 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:46 smithi150 ceph-mon[32290]: Deploying daemon mds.h on smithi150 2024-04-26T21:15:46.542 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:46 smithi150 ceph-mon[32290]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:46.542 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:46 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1978494542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:46.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:46 smithi171 ceph-mon[33478]: Deploying daemon mds.h on smithi150 2024-04-26T21:15:46.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:46 smithi171 ceph-mon[33478]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:46.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:46 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1978494542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:46.883 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:15:46 smithi150 systemd[1]: Starting Ceph mds.h for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:15:47.218 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:15:46 smithi150 podman[62633]: 2024-04-26 21:15:46.842144654 +0000 UTC m=+0.019006951 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:15:47.218 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:15:46 smithi150 podman[62633]: 2024-04-26 21:15:46.964454263 +0000 UTC m=+0.141316553 container create 0c77778bd39642c1e37cbcf8c7f839dfa85458c8e83bb3201a59e10e0981b2bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:15:47.633 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:15:47 smithi150 podman[62633]: 2024-04-26 21:15:47.217547168 +0000 UTC m=+0.394409462 container init 0c77778bd39642c1e37cbcf8c7f839dfa85458c8e83bb3201a59e10e0981b2bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:15:47.633 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:15:47 smithi150 podman[62633]: 2024-04-26 21:15:47.220476317 +0000 UTC m=+0.397338617 container start 0c77778bd39642c1e37cbcf8c7f839dfa85458c8e83bb3201a59e10e0981b2bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T21:15:47.633 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:15:47 smithi150 ceph-mds[62652]: -- 172.21.15.150:0/2699270012 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557023c95a40 con 0x557023c68c00 2024-04-26T21:15:47.633 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:15:47 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-h[62648]: starting mds.h at 2024-04-26T21:15:47.633 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:15:47 smithi150 bash[62633]: 0c77778bd39642c1e37cbcf8c7f839dfa85458c8e83bb3201a59e10e0981b2bb 2024-04-26T21:15:47.633 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:15:47 smithi150 systemd[1]: Started Ceph mds.h for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:48.149 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:48.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.258+0000 7f8119d97640 1 -- 172.21.15.129:0/1398864404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 msgr2=0x7f811410aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.258+0000 7f8119d97640 1 --2- 172.21.15.129:0/1398864404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 0x7f811410aaa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f810c009ca0 tx=0x7f810c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.258+0000 7f8119d97640 1 -- 172.21.15.129:0/1398864404 shutdown_connections 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.258+0000 7f8119d97640 1 --2- 172.21.15.129:0/1398864404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8114105d80 0x7f8114108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.258+0000 7f8119d97640 1 --2- 172.21.15.129:0/1398864404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 0x7f811410aaa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.258+0000 7f8119d97640 1 --2- 172.21.15.129:0/1398864404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8114103450 0x7f8114105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.258+0000 7f8119d97640 1 -- 172.21.15.129:0/1398864404 >> 172.21.15.129:0/1398864404 conn(0x7f81140fcde0 msgr2=0x7f81140ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.258+0000 7f8119d97640 1 -- 172.21.15.129:0/1398864404 shutdown_connections 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.258+0000 7f8119d97640 1 -- 172.21.15.129:0/1398864404 wait complete. 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8119d97640 1 Processor -- start 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8119d97640 1 -- start start 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8119d97640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8114103450 0x7f8114198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8119d97640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8114105d80 0x7f81141990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8119d97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 0x7f81141a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8119d97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f811410e4a0 con 0x7f81141086b0 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8119d97640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f811410e320 con 0x7f8114103450 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8119d97640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f811410e620 con 0x7f8114105d80 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8113fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 0x7f81141a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8113fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 0x7f81141a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34250/0 (socket says 172.21.15.129:34250) 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8113fff640 1 -- 172.21.15.129:0/2438531910 learned_addr learned my addr 172.21.15.129:0/2438531910 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f8112ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8114105d80 0x7f81141990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.259+0000 7f81137fe640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8114103450 0x7f8114198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8113fff640 1 -- 172.21.15.129:0/2438531910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8114105d80 msgr2=0x7f81141990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8113fff640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8114105d80 0x7f81141990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8113fff640 1 -- 172.21.15.129:0/2438531910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8114103450 msgr2=0x7f8114198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8113fff640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8114103450 0x7f8114198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8113fff640 1 -- 172.21.15.129:0/2438531910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81141a07c0 con 0x7f81141086b0 2024-04-26T21:15:48.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8113fff640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 0x7f81141a0150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f810c0097b0 tx=0x7f810c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:48.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8110ff9640 1 -- 172.21.15.129:0/2438531910 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c004240 con 0x7f81141086b0 2024-04-26T21:15:48.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8110ff9640 1 -- 172.21.15.129:0/2438531910 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810c0317c0 con 0x7f81141086b0 2024-04-26T21:15:48.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81141a0a50 con 0x7f81141086b0 2024-04-26T21:15:48.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8110ff9640 1 -- 172.21.15.129:0/2438531910 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c02fd80 con 0x7f81141086b0 2024-04-26T21:15:48.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.260+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141a0fa0 con 0x7f81141086b0 2024-04-26T21:15:48.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.261+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81140fe8d0 con 0x7f81141086b0 2024-04-26T21:15:48.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.262+0000 7f8110ff9640 1 -- 172.21.15.129:0/2438531910 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f810c0043e0 con 0x7f81141086b0 2024-04-26T21:15:48.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.263+0000 7f8110ff9640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f80fc0a3bb0 0x7f80fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.263+0000 7f8110ff9640 1 -- 172.21.15.129:0/2438531910 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f810c0f18b0 con 0x7f81141086b0 2024-04-26T21:15:48.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.263+0000 7f81137fe640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f80fc0a3bb0 0x7f80fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.263+0000 7f81137fe640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f80fc0a3bb0 0x7f80fc0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f810400bbe0 tx=0x7f810400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:48.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.266+0000 7f8110ff9640 1 -- 172.21.15.129:0/2438531910 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f810c0be850 con 0x7f81141086b0 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.150:6836/1663379365,v1:172.21.15.150:6837/1663379365] up:boot 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: fsmap 8 up:standby 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:48.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:48 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:48.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.482+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f811402dbf0 con 0x7f81141086b0 2024-04-26T21:15:48.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.483+0000 7f8110ff9640 1 -- 172.21.15.129:0/2438531910 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7f810c0be150 con 0x7f81141086b0 2024-04-26T21:15:48.484 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:48.484 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:15:48.484 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 9 2024-04-26T21:15:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f80fc0a3bb0 msgr2=0x7f80fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f80fc0a3bb0 0x7f80fc0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f810400bbe0 tx=0x7f810400b390 comp rx=0 tx=0).stop 2024-04-26T21:15:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 msgr2=0x7f81141a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 0x7f81141a0150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f810c0097b0 tx=0x7f810c003c20 comp rx=0 tx=0).stop 2024-04-26T21:15:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 shutdown_connections 2024-04-26T21:15:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f80fc0a3bb0 0x7f80fc0a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81141086b0 0x7f81141a0150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8114105d80 0x7f81141990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 --2- 172.21.15.129:0/2438531910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8114103450 0x7f8114198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 >> 172.21.15.129:0/2438531910 conn(0x7f81140fcde0 msgr2=0x7f8114103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 shutdown_connections 2024-04-26T21:15:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:48.485+0000 7f8119d97640 1 -- 172.21.15.129:0/2438531910 wait complete. 2024-04-26T21:15:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.150:6836/1663379365,v1:172.21.15.150:6837/1663379365] up:boot 2024-04-26T21:15:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: fsmap 8 up:standby 2024-04-26T21:15:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:15:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:48.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:48 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:48.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:48.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.150:6836/1663379365,v1:172.21.15.150:6837/1663379365] up:boot 2024-04-26T21:15:48.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: fsmap 8 up:standby 2024-04-26T21:15:48.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:15:48.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:48.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:48.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:48.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:48.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:48.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:48 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:49.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:49 smithi150 ceph-mon[32290]: Deploying daemon mds.i on smithi171 2024-04-26T21:15:49.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:49 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2438531910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:49.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:49 smithi129 ceph-mon[29237]: Deploying daemon mds.i on smithi171 2024-04-26T21:15:49.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:49 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2438531910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:49.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:49 smithi171 ceph-mon[33478]: Deploying daemon mds.i on smithi171 2024-04-26T21:15:49.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:49 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2438531910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:49.742 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:15:49 smithi171 systemd[1]: Starting Ceph mds.i for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:15:50.071 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:15:49 smithi171 podman[64460]: 2024-04-26 21:15:49.718952124 +0000 UTC m=+0.018443605 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:15:50.071 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:15:49 smithi171 podman[64460]: 2024-04-26 21:15:49.843196948 +0000 UTC m=+0.142688447 container create 2f80c7340542002418e51c936aeb3ea97a56facb7803a86c9dcf58a70c71645c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T21:15:50.071 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:15:50 smithi171 podman[64460]: 2024-04-26 21:15:50.07072621 +0000 UTC m=+0.370217667 container init 2f80c7340542002418e51c936aeb3ea97a56facb7803a86c9dcf58a70c71645c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T21:15:50.366 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:15:50 smithi171 podman[64460]: 2024-04-26 21:15:50.073457083 +0000 UTC m=+0.372948541 container start 2f80c7340542002418e51c936aeb3ea97a56facb7803a86c9dcf58a70c71645c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T21:15:50.366 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:15:50 smithi171 ceph-mds[64479]: -- 172.21.15.171:0/1692095246 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d48d1da40 con 0x561d48cf1000 2024-04-26T21:15:50.366 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:15:50 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i[64475]: starting mds.i at 2024-04-26T21:15:50.366 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:15:50 smithi171 bash[64460]: 2f80c7340542002418e51c936aeb3ea97a56facb7803a86c9dcf58a70c71645c 2024-04-26T21:15:50.367 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:15:50 smithi171 systemd[1]: Started Ceph mds.i for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:50.498 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.598+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/4250125506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64105d20 msgr2=0x7f3a64108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.598+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/4250125506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64105d20 0x7f3a64108110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009900 tx=0x7f3a6002a410 comp rx=0 tx=0).stop 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.598+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/4250125506 shutdown_connections 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.598+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/4250125506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a64108650 0x7f3a6410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.598+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/4250125506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64105d20 0x7f3a64108110 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.598+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/4250125506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a641033f0 0x7f3a641057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.598+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/4250125506 >> 172.21.15.129:0/4250125506 conn(0x7f3a640fcde0 msgr2=0x7f3a640ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.598+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/4250125506 shutdown_connections 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.598+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/4250125506 wait complete. 2024-04-26T21:15:50.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6c5e4640 1 Processor -- start 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6c5e4640 1 -- start start 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6c5e4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a641033f0 0x7f3a64198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6c5e4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a64105d20 0x7f3a64199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6c5e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64108650 0x7f3a641a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6c5e4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410e490 con 0x7f3a64108650 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6c5e4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410e310 con 0x7f3a641033f0 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6c5e4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410e610 con 0x7f3a64105d20 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6ab81640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64108650 0x7f3a641a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6ab81640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64108650 0x7f3a641a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34266/0 (socket says 172.21.15.129:34266) 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6ab81640 1 -- 172.21.15.129:0/2760839446 learned_addr learned my addr 172.21.15.129:0/2760839446 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a69b7f640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a64105d20 0x7f3a64199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6ab81640 1 -- 172.21.15.129:0/2760839446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a64105d20 msgr2=0x7f3a64199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6ab81640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a64105d20 0x7f3a64199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6a380640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a641033f0 0x7f3a64198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6ab81640 1 -- 172.21.15.129:0/2760839446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a641033f0 msgr2=0x7f3a64198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6ab81640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a641033f0 0x7f3a64198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6ab81640 1 -- 172.21.15.129:0/2760839446 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a641a0850 con 0x7f3a64108650 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.599+0000 7f3a6ab81640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64108650 0x7f3a641a01b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00eb10 tx=0x7f3a5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.600+0000 7f3a577fe640 1 -- 172.21.15.129:0/2760839446 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c004b20 con 0x7f3a64108650 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.600+0000 7f3a577fe640 1 -- 172.21.15.129:0/2760839446 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a5c004cc0 con 0x7f3a64108650 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.600+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a641a0b40 con 0x7f3a64108650 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.600+0000 7f3a577fe640 1 -- 172.21.15.129:0/2760839446 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c010670 con 0x7f3a64108650 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.600+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a641a1090 con 0x7f3a64108650 2024-04-26T21:15:50.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.601+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a640fe8d0 con 0x7f3a64108650 2024-04-26T21:15:50.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.602+0000 7f3a577fe640 1 -- 172.21.15.129:0/2760839446 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3a5c005020 con 0x7f3a64108650 2024-04-26T21:15:50.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.603+0000 7f3a577fe640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3a500a3c30 0x7f3a500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.603+0000 7f3a6a380640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3a500a3c30 0x7f3a500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.603+0000 7f3a577fe640 1 -- 172.21.15.129:0/2760839446 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3a5c0d0580 con 0x7f3a64108650 2024-04-26T21:15:50.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.603+0000 7f3a6a380640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3a500a3c30 0x7f3a500a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3a6419a110 tx=0x7f3a58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:50.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.607+0000 7f3a577fe640 1 -- 172.21.15.129:0/2760839446 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3a5c09d400 con 0x7f3a64108650 2024-04-26T21:15:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:50 smithi150 ceph-mon[32290]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:50 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:50 smithi129 ceph-mon[29237]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:50 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:50 smithi171 ceph-mon[33478]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:50.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:50.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:50.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:50 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:50.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.832+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a6402dad0 con 0x7f3a64108650 2024-04-26T21:15:50.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.833+0000 7f3a577fe640 1 -- 172.21.15.129:0/2760839446 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7f3a6402dad0 con 0x7f3a64108650 2024-04-26T21:15:50.834 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:50.834 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T21:15:50.834 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 10 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3a500a3c30 msgr2=0x7f3a500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3a500a3c30 0x7f3a500a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3a6419a110 tx=0x7f3a58005f90 comp rx=0 tx=0).stop 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64108650 msgr2=0x7f3a641a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64108650 0x7f3a641a01b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00eb10 tx=0x7f3a5c00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 shutdown_connections 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3a500a3c30 0x7f3a500a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a64108650 0x7f3a641a01b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a64105d20 0x7f3a64199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 --2- 172.21.15.129:0/2760839446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a641033f0 0x7f3a64198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 >> 172.21.15.129:0/2760839446 conn(0x7f3a640fcde0 msgr2=0x7f3a641052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 shutdown_connections 2024-04-26T21:15:50.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:50.835+0000 7f3a6c5e4640 1 -- 172.21.15.129:0/2760839446 wait complete. 2024-04-26T21:15:51.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:51 smithi129 ceph-mon[29237]: Deploying daemon mds.g on smithi129 2024-04-26T21:15:51.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:51 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.171:6838/2157967831,v1:172.21.15.171:6839/2157967831] up:boot 2024-04-26T21:15:51.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:51 smithi129 ceph-mon[29237]: fsmap 9 up:standby 2024-04-26T21:15:51.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:51 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:15:51.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:51 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2760839446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:51 smithi150 ceph-mon[32290]: Deploying daemon mds.g on smithi129 2024-04-26T21:15:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:51 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.171:6838/2157967831,v1:172.21.15.171:6839/2157967831] up:boot 2024-04-26T21:15:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:51 smithi150 ceph-mon[32290]: fsmap 9 up:standby 2024-04-26T21:15:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:51 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:15:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:51 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2760839446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:51.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:51 smithi171 ceph-mon[33478]: Deploying daemon mds.g on smithi129 2024-04-26T21:15:51.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:51 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.171:6838/2157967831,v1:172.21.15.171:6839/2157967831] up:boot 2024-04-26T21:15:51.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:51 smithi171 ceph-mon[33478]: fsmap 9 up:standby 2024-04-26T21:15:51.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:51 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:15:51.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:51 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2760839446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:52.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:52 smithi129 ceph-mon[29237]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:52.470 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:15:52 smithi129 systemd[1]: Starting Ceph mds.g for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:15:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:52 smithi150 ceph-mon[32290]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:52.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:52 smithi171 ceph-mon[33478]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:52.849 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:52.871 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:15:52 smithi129 podman[70603]: 2024-04-26 21:15:52.501551756 +0000 UTC m=+0.018620082 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:15:52.871 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:15:52 smithi129 podman[70603]: 2024-04-26 21:15:52.639989409 +0000 UTC m=+0.157057722 container create 74fb377ae3ac27128fa75581cf0b896f169ac018c74f9902416367ef46b5c85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T21:15:52.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.952+0000 7f8531f7f640 1 -- 172.21.15.129:0/4200130715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 msgr2=0x7f852c104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.952+0000 7f8531f7f640 1 --2- 172.21.15.129:0/4200130715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 0x7f852c104010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f851c009900 tx=0x7f851c02a410 comp rx=0 tx=0).stop 2024-04-26T21:15:52.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 -- 172.21.15.129:0/4200130715 shutdown_connections 2024-04-26T21:15:52.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 --2- 172.21.15.129:0/4200130715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c104550 0x7f852c10ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 --2- 172.21.15.129:0/4200130715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 0x7f852c104010 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 --2- 172.21.15.129:0/4200130715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c1029a0 0x7f852c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 -- 172.21.15.129:0/4200130715 >> 172.21.15.129:0/4200130715 conn(0x7f852c0fdf10 msgr2=0x7f852c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 -- 172.21.15.129:0/4200130715 shutdown_connections 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 -- 172.21.15.129:0/4200130715 wait complete. 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 Processor -- start 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 -- start start 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c1029a0 0x7f852c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 0x7f852c199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c104550 0x7f852c1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f852c10e4b0 con 0x7f852c103b90 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f852c10e330 con 0x7f852c1029a0 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.953+0000 7f8531f7f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f852c10e630 con 0x7f852c104550 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 0x7f852c199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 0x7f852c199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45372/0 (socket says 172.21.15.129:45372) 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852affd640 1 -- 172.21.15.129:0/3988245942 learned_addr learned my addr 172.21.15.129:0/3988245942 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852b7fe640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c1029a0 0x7f852c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852bfff640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c104550 0x7f852c1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852affd640 1 -- 172.21.15.129:0/3988245942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c104550 msgr2=0x7f852c1a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852affd640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c104550 0x7f852c1a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852affd640 1 -- 172.21.15.129:0/3988245942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c1029a0 msgr2=0x7f852c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852affd640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c1029a0 0x7f852c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852affd640 1 -- 172.21.15.129:0/3988245942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c1a0710 con 0x7f852c103b90 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852b7fe640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c1029a0 0x7f852c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f852affd640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 0x7f852c199030 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f851c031500 tx=0x7f851c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f8528ff9640 1 -- 172.21.15.129:0/3988245942 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c03f030 con 0x7f852c103b90 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852c1a09a0 con 0x7f852c103b90 2024-04-26T21:15:52.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f8528ff9640 1 -- 172.21.15.129:0/3988245942 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851c003800 con 0x7f852c103b90 2024-04-26T21:15:52.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f8528ff9640 1 -- 172.21.15.129:0/3988245942 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c035070 con 0x7f852c103b90 2024-04-26T21:15:52.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.954+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852c1a0e90 con 0x7f852c103b90 2024-04-26T21:15:52.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.955+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c109cc0 con 0x7f852c103b90 2024-04-26T21:15:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.956+0000 7f8528ff9640 1 -- 172.21.15.129:0/3988245942 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f851c0039b0 con 0x7f852c103b90 2024-04-26T21:15:52.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.957+0000 7f8528ff9640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85140a3bb0 0x7f85140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.957+0000 7f852b7fe640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85140a3bb0 0x7f85140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.957+0000 7f8528ff9640 1 -- 172.21.15.129:0/3988245942 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f851c0f1080 con 0x7f852c103b90 2024-04-26T21:15:52.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.957+0000 7f852b7fe640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85140a3bb0 0x7f85140a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f852c1039f0 tx=0x7f852000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:52.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:52.961+0000 7f8528ff9640 1 -- 172.21.15.129:0/3988245942 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f851c0bdf00 con 0x7f852c103b90 2024-04-26T21:15:53.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.187+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f852c02dbf0 con 0x7f852c103b90 2024-04-26T21:15:53.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.187+0000 7f8528ff9640 1 -- 172.21.15.129:0/3988245942 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7f851c0bd800 con 0x7f852c103b90 2024-04-26T21:15:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:53.189 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T21:15:53.189 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 10 2024-04-26T21:15:53.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85140a3bb0 msgr2=0x7f85140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85140a3bb0 0x7f85140a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f852c1039f0 tx=0x7f852000a5e0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 msgr2=0x7f852c199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 0x7f852c199030 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f851c031500 tx=0x7f851c0043c0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 shutdown_connections 2024-04-26T21:15:53.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85140a3bb0 0x7f85140a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c104550 0x7f852c1a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c103b90 0x7f852c199030 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 --2- 172.21.15.129:0/3988245942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c1029a0 0x7f852c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 >> 172.21.15.129:0/3988245942 conn(0x7f852c0fdf10 msgr2=0x7f852c106e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:53.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 shutdown_connections 2024-04-26T21:15:53.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:53.190+0000 7f8531f7f640 1 -- 172.21.15.129:0/3988245942 wait complete. 2024-04-26T21:15:53.201 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:15:52 smithi129 podman[70603]: 2024-04-26 21:15:52.903261165 +0000 UTC m=+0.420329479 container init 74fb377ae3ac27128fa75581cf0b896f169ac018c74f9902416367ef46b5c85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:15:53.202 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:15:52 smithi129 podman[70603]: 2024-04-26 21:15:52.906466808 +0000 UTC m=+0.423535142 container start 74fb377ae3ac27128fa75581cf0b896f169ac018c74f9902416367ef46b5c85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T21:15:53.202 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:15:52 smithi129 ceph-mds[70642]: -- 172.21.15.129:0/3762877112 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc2a883a40 con 0x55fc2a857000 2024-04-26T21:15:53.202 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:15:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g[70618]: starting mds.g at 2024-04-26T21:15:53.202 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:15:53 smithi129 bash[70603]: 74fb377ae3ac27128fa75581cf0b896f169ac018c74f9902416367ef46b5c85b 2024-04-26T21:15:53.202 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:15:53 smithi129 systemd[1]: Started Ceph mds.g for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:54.291 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:54.359 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:54.359 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:54.359 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:54.359 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:54.359 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:54.359 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: Deploying daemon mds.a on smithi129 2024-04-26T21:15:54.360 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3988245942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:54.360 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:54.360 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.129:6838/556190328,v1:172.21.15.129:6839/556190328] up:boot 2024-04-26T21:15:54.360 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: fsmap 10 up:standby 2024-04-26T21:15:54.360 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:54 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:15:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: Deploying daemon mds.a on smithi129 2024-04-26T21:15:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3988245942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:54.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.129:6838/556190328,v1:172.21.15.129:6839/556190328] up:boot 2024-04-26T21:15:54.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: fsmap 10 up:standby 2024-04-26T21:15:54.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:54 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:15:54.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: Deploying daemon mds.a on smithi129 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3988245942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.129:6838/556190328,v1:172.21.15.129:6839/556190328] up:boot 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: fsmap 10 up:standby 2024-04-26T21:15:54.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:54 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:15:55.204 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:55.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.373+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/2859151966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78103c00 msgr2=0x7f0c78104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.373+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/2859151966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78103c00 0x7f0c78104080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009930 tx=0x7f0c6002a420 comp rx=0 tx=0).stop 2024-04-26T21:15:55.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.373+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/2859151966 shutdown_connections 2024-04-26T21:15:55.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.373+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/2859151966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c781045c0 0x7f0c7810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.373+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/2859151966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78103c00 0x7f0c78104080 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.373+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/2859151966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c78100c30 0x7f0c78101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.373+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/2859151966 >> 172.21.15.129:0/2859151966 conn(0x7f0c780fc560 msgr2=0x7f0c780fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.373+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/2859151966 shutdown_connections 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.373+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/2859151966 wait complete. 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c7dc2a640 1 Processor -- start 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c7dc2a640 1 -- start start 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c7dc2a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78100c30 0x7f0c78198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c7dc2a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c78103c00 0x7f0c78199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c7dc2a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c781045c0 0x7f0c781a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c7dc2a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810e510 con 0x7f0c78100c30 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c7dc2a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810e390 con 0x7f0c781045c0 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c7dc2a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810e690 con 0x7f0c78103c00 2024-04-26T21:15:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78100c30 0x7f0c78198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78100c30 0x7f0c78198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45388/0 (socket says 172.21.15.129:45388) 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c777fe640 1 -- 172.21.15.129:0/1923110645 learned_addr learned my addr 172.21.15.129:0/1923110645 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c777fe640 1 -- 172.21.15.129:0/1923110645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c78103c00 msgr2=0x7f0c78199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c777fe640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c78103c00 0x7f0c78199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c777fe640 1 -- 172.21.15.129:0/1923110645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c781045c0 msgr2=0x7f0c781a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c777fe640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c781045c0 0x7f0c781a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.374+0000 7f0c777fe640 1 -- 172.21.15.129:0/1923110645 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c781a0840 con 0x7f0c78100c30 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.375+0000 7f0c777fe640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78100c30 0x7f0c78198c30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d770 tx=0x7f0c6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.375+0000 7f0c757fa640 1 -- 172.21.15.129:0/1923110645 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68017070 con 0x7f0c78100c30 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.375+0000 7f0c757fa640 1 -- 172.21.15.129:0/1923110645 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c68009c30 con 0x7f0c78100c30 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.375+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c781a0b30 con 0x7f0c78100c30 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.375+0000 7f0c757fa640 1 -- 172.21.15.129:0/1923110645 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68003cc0 con 0x7f0c78100c30 2024-04-26T21:15:55.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.375+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781a1100 con 0x7f0c78100c30 2024-04-26T21:15:55.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.376+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c78109db0 con 0x7f0c78100c30 2024-04-26T21:15:55.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.377+0000 7f0c757fa640 1 -- 172.21.15.129:0/1923110645 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0c68003e60 con 0x7f0c78100c30 2024-04-26T21:15:55.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.378+0000 7f0c757fa640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3bb0 0x7f0c640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.378+0000 7f0c6effd640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3bb0 0x7f0c640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.378+0000 7f0c757fa640 1 -- 172.21.15.129:0/1923110645 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0c680cfa30 con 0x7f0c78100c30 2024-04-26T21:15:55.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.378+0000 7f0c6effd640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3bb0 0x7f0c640a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c600096c0 tx=0x7f0c60000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:55.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.383+0000 7f0c757fa640 1 -- 172.21.15.129:0/1923110645 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0c6809c8b0 con 0x7f0c78100c30 2024-04-26T21:15:55.470 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:15:55 smithi129 systemd[1]: Starting Ceph mds.a for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:15:55.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.618+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c7802dd10 con 0x7f0c78100c30 2024-04-26T21:15:55.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.619+0000 7f0c757fa640 1 -- 172.21.15.129:0/1923110645 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8845 (secure 0 0 0) 0x7f0c68009de0 con 0x7f0c78100c30 2024-04-26T21:15:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:55.620 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T21:15:55.620 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 11 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3bb0 msgr2=0x7f0c640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3bb0 0x7f0c640a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c600096c0 tx=0x7f0c60000bc0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78100c30 msgr2=0x7f0c78198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78100c30 0x7f0c78198c30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d770 tx=0x7f0c6800dc30 comp rx=0 tx=0).stop 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 shutdown_connections 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3bb0 0x7f0c640a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c781045c0 0x7f0c781a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c78103c00 0x7f0c78199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 --2- 172.21.15.129:0/1923110645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c78100c30 0x7f0c78198c30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 >> 172.21.15.129:0/1923110645 conn(0x7f0c780fc560 msgr2=0x7f0c78106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:55.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 shutdown_connections 2024-04-26T21:15:55.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:55.621+0000 7f0c7dc2a640 1 -- 172.21.15.129:0/1923110645 wait complete. 2024-04-26T21:15:55.959 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:15:55 smithi129 podman[71006]: 2024-04-26 21:15:55.442921489 +0000 UTC m=+0.019210143 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:15:55.959 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:15:55 smithi129 podman[71006]: 2024-04-26 21:15:55.571833188 +0000 UTC m=+0.148121828 container create 244246ed44dad71cf235050e08a7649f6bfb046c1b79dfe2320ddefeaca3885b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T21:15:55.959 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:15:55 smithi129 podman[71006]: 2024-04-26 21:15:55.958760186 +0000 UTC m=+0.535048825 container init 244246ed44dad71cf235050e08a7649f6bfb046c1b79dfe2320ddefeaca3885b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T21:15:56.220 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:15:55 smithi129 podman[71006]: 2024-04-26 21:15:55.96140278 +0000 UTC m=+0.537691420 container start 244246ed44dad71cf235050e08a7649f6bfb046c1b79dfe2320ddefeaca3885b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T21:15:56.220 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:15:55 smithi129 ceph-mds[71027]: -- 172.21.15.129:0/3261816054 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626defa1a40 con 0x5626def75000 2024-04-26T21:15:56.220 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:15:55 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a[71023]: starting mds.a at 2024-04-26T21:15:56.220 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:15:56 smithi129 bash[71006]: 244246ed44dad71cf235050e08a7649f6bfb046c1b79dfe2320ddefeaca3885b 2024-04-26T21:15:56.220 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:15:56 smithi129 systemd[1]: Started Ceph mds.a for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:56 smithi150 ceph-mon[32290]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:56 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1923110645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:56 smithi129 ceph-mon[29237]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:56 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1923110645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:56 smithi171 ceph-mon[33478]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:56 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1923110645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:15:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:15:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:15:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:57 smithi150 ceph-mon[32290]: Deploying daemon mds.b on smithi150 2024-04-26T21:15:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:57 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.129:6840/861528806,v1:172.21.15.129:6841/861528806] up:boot 2024-04-26T21:15:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:57 smithi150 ceph-mon[32290]: fsmap 11 up:standby 2024-04-26T21:15:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:57 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:15:57.633 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:57.660 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:57 smithi129 ceph-mon[29237]: Deploying daemon mds.b on smithi150 2024-04-26T21:15:57.660 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:57 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.129:6840/861528806,v1:172.21.15.129:6841/861528806] up:boot 2024-04-26T21:15:57.660 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:57 smithi129 ceph-mon[29237]: fsmap 11 up:standby 2024-04-26T21:15:57.660 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:57 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:15:57.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:57 smithi171 ceph-mon[33478]: Deploying daemon mds.b on smithi150 2024-04-26T21:15:57.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:57 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.129:6840/861528806,v1:172.21.15.129:6841/861528806] up:boot 2024-04-26T21:15:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:57 smithi171 ceph-mon[33478]: fsmap 11 up:standby 2024-04-26T21:15:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:57 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:15:57.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.746+0000 7faa7e464640 1 -- 172.21.15.129:0/3138700732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa78102a60 msgr2=0x7faa78102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.746+0000 7faa7e464640 1 --2- 172.21.15.129:0/3138700732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa78102a60 0x7faa78102ee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009900 tx=0x7faa6c02a410 comp rx=0 tx=0).stop 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.746+0000 7faa7e464640 1 -- 172.21.15.129:0/3138700732 shutdown_connections 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.746+0000 7faa7e464640 1 --2- 172.21.15.129:0/3138700732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa78103420 0x7faa78109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.746+0000 7faa7e464640 1 --2- 172.21.15.129:0/3138700732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa78102a60 0x7faa78102ee0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.746+0000 7faa7e464640 1 --2- 172.21.15.129:0/3138700732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa78101870 0x7faa78101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.746+0000 7faa7e464640 1 -- 172.21.15.129:0/3138700732 >> 172.21.15.129:0/3138700732 conn(0x7faa780fcde0 msgr2=0x7faa780ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.746+0000 7faa7e464640 1 -- 172.21.15.129:0/3138700732 shutdown_connections 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.746+0000 7faa7e464640 1 -- 172.21.15.129:0/3138700732 wait complete. 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa7e464640 1 Processor -- start 2024-04-26T21:15:57.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa7e464640 1 -- start start 2024-04-26T21:15:57.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa7e464640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa78101870 0x7faa78198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa7e464640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa78102a60 0x7faa781990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa7e464640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa78103420 0x7faa781a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa7e464640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa7810d3d0 con 0x7faa78102a60 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa7e464640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faa7810d250 con 0x7faa78103420 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa7e464640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faa7810d550 con 0x7faa78101870 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa78102a60 0x7faa781990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa78102a60 0x7faa781990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45430/0 (socket says 172.21.15.129:45430) 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa777fe640 1 -- 172.21.15.129:0/1553749398 learned_addr learned my addr 172.21.15.129:0/1553749398 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa77fff640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa78101870 0x7faa78198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.747+0000 7faa7ca01640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa78103420 0x7faa781a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa777fe640 1 -- 172.21.15.129:0/1553749398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa78101870 msgr2=0x7faa78198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa777fe640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa78101870 0x7faa78198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa777fe640 1 -- 172.21.15.129:0/1553749398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa78103420 msgr2=0x7faa781a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa777fe640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa78103420 0x7faa781a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa777fe640 1 -- 172.21.15.129:0/1553749398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa781a07d0 con 0x7faa78102a60 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa777fe640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa78102a60 0x7faa781990f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faa6c005ab0 tx=0x7faa6c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa757fa640 1 -- 172.21.15.129:0/1553749398 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c03f070 con 0x7faa78102a60 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa781a0a60 con 0x7faa78102a60 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa757fa640 1 -- 172.21.15.129:0/1553749398 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa6c031c50 con 0x7faa78102a60 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa757fa640 1 -- 172.21.15.129:0/1553749398 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c035030 con 0x7faa78102a60 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.748+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa781a0fa0 con 0x7faa78102a60 2024-04-26T21:15:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.749+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa781a1290 con 0x7faa78102a60 2024-04-26T21:15:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.750+0000 7faa757fa640 1 -- 172.21.15.129:0/1553749398 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7faa6c042070 con 0x7faa78102a60 2024-04-26T21:15:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.751+0000 7faa757fa640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faa600a3bb0 0x7faa600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.751+0000 7faa77fff640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faa600a3bb0 0x7faa600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.751+0000 7faa757fa640 1 -- 172.21.15.129:0/1553749398 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7faa6c0f0660 con 0x7faa78102a60 2024-04-26T21:15:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.751+0000 7faa77fff640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faa600a3bb0 0x7faa600a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa781028c0 tx=0x7faa68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:57.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.756+0000 7faa757fa640 1 -- 172.21.15.129:0/1553749398 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faa6c0bce90 con 0x7faa78102a60 2024-04-26T21:15:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.968+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa781a1460 con 0x7faa78102a60 2024-04-26T21:15:57.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.969+0000 7faa757fa640 1 -- 172.21.15.129:0/1553749398 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9671 (secure 0 0 0) 0x7faa6c0c1190 con 0x7faa78102a60 2024-04-26T21:15:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:15:57.970 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T21:15:57.971 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 12 2024-04-26T21:15:57.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faa600a3bb0 msgr2=0x7faa600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faa600a3bb0 0x7faa600a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa781028c0 tx=0x7faa68005f90 comp rx=0 tx=0).stop 2024-04-26T21:15:57.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa78102a60 msgr2=0x7faa781990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa78102a60 0x7faa781990f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faa6c005ab0 tx=0x7faa6c003b90 comp rx=0 tx=0).stop 2024-04-26T21:15:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 shutdown_connections 2024-04-26T21:15:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faa600a3bb0 0x7faa600a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa78103420 0x7faa781a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa78102a60 0x7faa781990f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 --2- 172.21.15.129:0/1553749398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa78101870 0x7faa78198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 >> 172.21.15.129:0/1553749398 conn(0x7faa780fcde0 msgr2=0x7faa780fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 shutdown_connections 2024-04-26T21:15:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:15:57.971+0000 7faa7e464640 1 -- 172.21.15.129:0/1553749398 wait complete. 2024-04-26T21:15:58.615 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:58 smithi150 ceph-mon[32290]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:58.615 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:15:58 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1553749398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:58.615 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:15:58 smithi150 systemd[1]: Starting Ceph mds.b for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:15:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:58 smithi129 ceph-mon[29237]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:15:58 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1553749398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:58 smithi171 ceph-mon[33478]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:15:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:15:58 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1553749398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:15:58.883 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:15:58 smithi150 podman[62940]: 2024-04-26 21:15:58.517064641 +0000 UTC m=+0.017942471 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:15:58.883 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:15:58 smithi150 podman[62940]: 2024-04-26 21:15:58.645874791 +0000 UTC m=+0.146752609 container create d3dd5bd2c80934d401ed39cd447180a81a868a563567695d0f0fd9e072b41554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T21:15:59.382 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:15:58 smithi150 podman[62940]: 2024-04-26 21:15:58.903903984 +0000 UTC m=+0.404781798 container init d3dd5bd2c80934d401ed39cd447180a81a868a563567695d0f0fd9e072b41554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-26T21:15:59.383 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:15:58 smithi150 podman[62940]: 2024-04-26 21:15:58.906470408 +0000 UTC m=+0.407348222 container start d3dd5bd2c80934d401ed39cd447180a81a868a563567695d0f0fd9e072b41554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T21:15:59.383 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:15:58 smithi150 ceph-mds[62959]: -- 172.21.15.150:0/2563342105 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56218176ba40 con 0x56218173ec00 2024-04-26T21:15:59.383 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:15:58 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b[62955]: starting mds.b at 2024-04-26T21:15:59.383 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:15:59 smithi150 bash[62940]: d3dd5bd2c80934d401ed39cd447180a81a868a563567695d0f0fd9e072b41554 2024-04-26T21:15:59.383 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:15:59 smithi150 systemd[1]: Started Ceph mds.b for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:15:59.985 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:16:00.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.085+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1135598985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88103b70 msgr2=0x7fcf88103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.085+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1135598985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88103b70 0x7fcf88103ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009900 tx=0x7fcf7802a410 comp rx=0 tx=0).stop 2024-04-26T21:16:00.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.085+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1135598985 shutdown_connections 2024-04-26T21:16:00.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.085+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1135598985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf88104530 0x7fcf8810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.085+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1135598985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88103b70 0x7fcf88103ff0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.085+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1135598985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf88102980 0x7fcf88102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.085+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1135598985 >> 172.21.15.129:0/1135598985 conn(0x7fcf880fdf30 msgr2=0x7fcf88100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.085+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1135598985 shutdown_connections 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.085+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1135598985 wait complete. 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8fd51640 1 Processor -- start 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8fd51640 1 -- start start 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8fd51640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88102980 0x7fcf88198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8fd51640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf88103b70 0x7fcf881990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8fd51640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf88104530 0x7fcf881a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8fd51640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8810e4b0 con 0x7fcf88102980 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8fd51640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8810e330 con 0x7fcf88103b70 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8fd51640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8810e630 con 0x7fcf88104530 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8daed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88102980 0x7fcf88198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8daed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88102980 0x7fcf88198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45438/0 (socket says 172.21.15.129:45438) 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8daed640 1 -- 172.21.15.129:0/1408836150 learned_addr learned my addr 172.21.15.129:0/1408836150 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8d2ec640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf88103b70 0x7fcf881990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8e2ee640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf88104530 0x7fcf881a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8daed640 1 -- 172.21.15.129:0/1408836150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf88104530 msgr2=0x7fcf881a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8daed640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf88104530 0x7fcf881a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8daed640 1 -- 172.21.15.129:0/1408836150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf88103b70 msgr2=0x7fcf881990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8daed640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf88103b70 0x7fcf881990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8daed640 1 -- 172.21.15.129:0/1408836150 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf881a07b0 con 0x7fcf88102980 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8d2ec640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf88103b70 0x7fcf881990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8e2ee640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf88104530 0x7fcf881a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8daed640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88102980 0x7fcf88198b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400c970 tx=0x7fcf8400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf7effd640 1 -- 172.21.15.129:0/1408836150 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84004220 con 0x7fcf88102980 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf881a0aa0 con 0x7fcf88102980 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf7effd640 1 -- 172.21.15.129:0/1408836150 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf840043c0 con 0x7fcf88102980 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.086+0000 7fcf7effd640 1 -- 172.21.15.129:0/1408836150 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84004a00 con 0x7fcf88102980 2024-04-26T21:16:00.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.087+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf881a0ff0 con 0x7fcf88102980 2024-04-26T21:16:00.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.087+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf88109d20 con 0x7fcf88102980 2024-04-26T21:16:00.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.088+0000 7fcf7effd640 1 -- 172.21.15.129:0/1408836150 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fcf840079b0 con 0x7fcf88102980 2024-04-26T21:16:00.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.089+0000 7fcf7effd640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf740a3bb0 0x7fcf740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.089+0000 7fcf8d2ec640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf740a3bb0 0x7fcf740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.089+0000 7fcf7effd640 1 -- 172.21.15.129:0/1408836150 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcf840cfd90 con 0x7fcf88102980 2024-04-26T21:16:00.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.089+0000 7fcf8d2ec640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf740a3bb0 0x7fcf740a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf8819a0a0 tx=0x7fcf780314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.094+0000 7fcf7effd640 1 -- 172.21.15.129:0/1408836150 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcf84014030 con 0x7fcf88102980 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] up:boot 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: fsmap 12 up:standby 2024-04-26T21:16:00.299 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:00 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:16:00.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.323+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf8802dcf0 con 0x7fcf88102980 2024-04-26T21:16:00.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.324+0000 7fcf7effd640 1 -- 172.21.15.129:0/1408836150 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10500 (secure 0 0 0) 0x7fcf8409cbf0 con 0x7fcf88102980 2024-04-26T21:16:00.325 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:00.325 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T21:16:00.326 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 13 2024-04-26T21:16:00.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf740a3bb0 msgr2=0x7fcf740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf740a3bb0 0x7fcf740a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf8819a0a0 tx=0x7fcf780314a0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88102980 msgr2=0x7fcf88198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88102980 0x7fcf88198b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400c970 tx=0x7fcf8400ce30 comp rx=0 tx=0).stop 2024-04-26T21:16:00.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 shutdown_connections 2024-04-26T21:16:00.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf740a3bb0 0x7fcf740a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf88103b70 0x7fcf881990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf88104530 0x7fcf881a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 --2- 172.21.15.129:0/1408836150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf88102980 0x7fcf88198b90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 >> 172.21.15.129:0/1408836150 conn(0x7fcf880fdf30 msgr2=0x7fcf88106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 shutdown_connections 2024-04-26T21:16:00.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.326+0000 7fcf8fd51640 1 -- 172.21.15.129:0/1408836150 wait complete. 2024-04-26T21:16:00.338 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] up:boot 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: fsmap 12 up:standby 2024-04-26T21:16:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:00 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] up:boot 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: fsmap 12 up:standby 2024-04-26T21:16:00.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:00 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:16:00.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 -- 172.21.15.129:0/3302409540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 msgr2=0x7f9cb8102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/3302409540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 0x7f9cb8102ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009900 tx=0x7f9ca002a410 comp rx=0 tx=0).stop 2024-04-26T21:16:00.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 -- 172.21.15.129:0/3302409540 shutdown_connections 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/3302409540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb8103420 0x7f9cb8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/3302409540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 0x7f9cb8102ee0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/3302409540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cb8101870 0x7f9cb8101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 -- 172.21.15.129:0/3302409540 >> 172.21.15.129:0/3302409540 conn(0x7f9cb80fcde0 msgr2=0x7f9cb80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 -- 172.21.15.129:0/3302409540 shutdown_connections 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 -- 172.21.15.129:0/3302409540 wait complete. 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 Processor -- start 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 -- start start 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb8101870 0x7f9cb8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 0x7f9cb8199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cb8103420 0x7f9cb81a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb810e4c0 con 0x7f9cb8102a60 2024-04-26T21:16:00.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb810e340 con 0x7f9cb8101870 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.439+0000 7f9cbd86d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb810e640 con 0x7f9cb8103420 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 0x7f9cb8199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 0x7f9cb8199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45456/0 (socket says 172.21.15.129:45456) 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb67fc640 1 -- 172.21.15.129:0/4252964298 learned_addr learned my addr 172.21.15.129:0/4252964298 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb77fe640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cb8103420 0x7f9cb81a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb6ffd640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb8101870 0x7f9cb8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb67fc640 1 -- 172.21.15.129:0/4252964298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cb8103420 msgr2=0x7f9cb81a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb67fc640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cb8103420 0x7f9cb81a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb67fc640 1 -- 172.21.15.129:0/4252964298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb8101870 msgr2=0x7f9cb8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb67fc640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb8101870 0x7f9cb8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb67fc640 1 -- 172.21.15.129:0/4252964298 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb81a0740 con 0x7f9cb8102a60 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cb67fc640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 0x7f9cb8199060 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0005940 tx=0x7f9ca0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cbc86b640 1 -- 172.21.15.129:0/4252964298 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca003f030 con 0x7f9cb8102a60 2024-04-26T21:16:00.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb81a09d0 con 0x7f9cb8102a60 2024-04-26T21:16:00.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cbc86b640 1 -- 172.21.15.129:0/4252964298 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ca0003aa0 con 0x7f9cb8102a60 2024-04-26T21:16:00.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cbc86b640 1 -- 172.21.15.129:0/4252964298 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0035070 con 0x7f9cb8102a60 2024-04-26T21:16:00.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.440+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb81a0f10 con 0x7f9cb8102a60 2024-04-26T21:16:00.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.441+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb8108b90 con 0x7f9cb8102a60 2024-04-26T21:16:00.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.442+0000 7f9cbc86b640 1 -- 172.21.15.129:0/4252964298 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9ca0042030 con 0x7f9cb8102a60 2024-04-26T21:16:00.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.443+0000 7f9cbc86b640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.443+0000 7f9cb6ffd640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.443+0000 7f9cbc86b640 1 -- 172.21.15.129:0/4252964298 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9ca00f04f0 con 0x7f9cb8102a60 2024-04-26T21:16:00.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.444+0000 7f9cb6ffd640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9cb81028c0 tx=0x7f9cb0007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.448+0000 7f9cbc86b640 1 -- 172.21.15.129:0/4252964298 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9ca00bcd20 con 0x7f9cb8102a60 2024-04-26T21:16:00.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.627+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cb802dca0 con 0x7f9cb8102a60 2024-04-26T21:16:00.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.628+0000 7f9cbc86b640 1 -- 172.21.15.129:0/4252964298 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18511 (secure 0 0 0) 0x7f9ca00c1020 con 0x7f9cb8102a60 2024-04-26T21:16:00.631 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:00.631 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":82,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:15:14.397240+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.630+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9ca40a3bb0 msgr2=0x7f9ca40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.630+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9cb81028c0 tx=0x7f9cb0007600 comp rx=0 tx=0).stop 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 msgr2=0x7f9cb8199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 0x7f9cb8199060 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0005940 tx=0x7f9ca0031a50 comp rx=0 tx=0).stop 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 shutdown_connections 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cb8103420 0x7f9cb81a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9cb8102a60 0x7f9cb8199060 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 --2- 172.21.15.129:0/4252964298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb8101870 0x7f9cb8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 >> 172.21.15.129:0/4252964298 conn(0x7f9cb80fcde0 msgr2=0x7f9cb8105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 shutdown_connections 2024-04-26T21:16:00.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.631+0000 7f9cbd86d640 1 -- 172.21.15.129:0/4252964298 wait complete. 2024-04-26T21:16:00.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:16:00.645 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:16:00.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.752+0000 7f8f14d97640 1 -- 172.21.15.129:0/4284939444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10101ac0 msgr2=0x7f8f10101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.752+0000 7f8f14d97640 1 --2- 172.21.15.129:0/4284939444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10101ac0 0x7f8f10101f40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009900 tx=0x7f8ef802a400 comp rx=0 tx=0).stop 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.752+0000 7f8f14d97640 1 -- 172.21.15.129:0/4284939444 shutdown_connections 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.752+0000 7f8f14d97640 1 --2- 172.21.15.129:0/4284939444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f10102480 0x7f8f10109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.752+0000 7f8f14d97640 1 --2- 172.21.15.129:0/4284939444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10101ac0 0x7f8f10101f40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.752+0000 7f8f14d97640 1 --2- 172.21.15.129:0/4284939444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f10102f00 0x7f8f10103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.752+0000 7f8f14d97640 1 -- 172.21.15.129:0/4284939444 >> 172.21.15.129:0/4284939444 conn(0x7f8f100fc570 msgr2=0x7f8f100fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.752+0000 7f8f14d97640 1 -- 172.21.15.129:0/4284939444 shutdown_connections 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.752+0000 7f8f14d97640 1 -- 172.21.15.129:0/4284939444 wait complete. 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f14d97640 1 Processor -- start 2024-04-26T21:16:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f14d97640 1 -- start start 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f14d97640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f10101ac0 0x7f8f10101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f14d97640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f10102480 0x7f8f1019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f14d97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10102f00 0x7f8f1019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f14d97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1010d3a0 con 0x7f8f10102f00 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f14d97640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1010d220 con 0x7f8f10102480 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f14d97640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1010d520 con 0x7f8f10101ac0 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f0ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10102f00 0x7f8f1019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f0e59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f10101ac0 0x7f8f10101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f0ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10102f00 0x7f8f1019ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45466/0 (socket says 172.21.15.129:45466) 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f0ed9d640 1 -- 172.21.15.129:0/316550360 learned_addr learned my addr 172.21.15.129:0/316550360 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.753+0000 7f8f07fff640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f10102480 0x7f8f1019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f0ed9d640 1 -- 172.21.15.129:0/316550360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f10101ac0 msgr2=0x7f8f10101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f0ed9d640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f10101ac0 0x7f8f10101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f0ed9d640 1 -- 172.21.15.129:0/316550360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f10102480 msgr2=0x7f8f1019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f0ed9d640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f10102480 0x7f8f1019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f0ed9d640 1 -- 172.21.15.129:0/316550360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f101a0860 con 0x7f8f10102f00 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f0ed9d640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10102f00 0x7f8f1019ffa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009fd0 tx=0x7f8f0800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f077fe640 1 -- 172.21.15.129:0/316550360 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08017070 con 0x7f8f10102f00 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f077fe640 1 -- 172.21.15.129:0/316550360 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f08003e20 con 0x7f8f10102f00 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f101a0b50 con 0x7f8f10102f00 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f077fe640 1 -- 172.21.15.129:0/316550360 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08010460 con 0x7f8f10102f00 2024-04-26T21:16:00.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.754+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f101a1020 con 0x7f8f10102f00 2024-04-26T21:16:00.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.755+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f10108d10 con 0x7f8f10102f00 2024-04-26T21:16:00.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.756+0000 7f8f077fe640 1 -- 172.21.15.129:0/316550360 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8f08003970 con 0x7f8f10102f00 2024-04-26T21:16:00.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.757+0000 7f8f077fe640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8efc0a3bb0 0x7f8efc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.757+0000 7f8f0e59c640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8efc0a3bb0 0x7f8efc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.757+0000 7f8f077fe640 1 -- 172.21.15.129:0/316550360 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8f080cfce0 con 0x7f8f10102f00 2024-04-26T21:16:00.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.757+0000 7f8f0e59c640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8efc0a3bb0 0x7f8efc0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009b50 tx=0x7f8f00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.762+0000 7f8f077fe640 1 -- 172.21.15.129:0/316550360 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8f0809c510 con 0x7f8f10102f00 2024-04-26T21:16:00.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.959+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f1002dc80 con 0x7f8f10102f00 2024-04-26T21:16:00.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.959+0000 7f8f077fe640 1 -- 172.21.15.129:0/316550360 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f8f080a0810 con 0x7f8f10102f00 2024-04-26T21:16:00.962 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8efc0a3bb0 msgr2=0x7f8efc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8efc0a3bb0 0x7f8efc0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009b50 tx=0x7f8f00005f90 comp rx=0 tx=0).stop 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10102f00 msgr2=0x7f8f1019ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10102f00 0x7f8f1019ffa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009fd0 tx=0x7f8f0800b260 comp rx=0 tx=0).stop 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 shutdown_connections 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8efc0a3bb0 0x7f8efc0a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f10102480 0x7f8f1019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f10102f00 0x7f8f1019ffa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 --2- 172.21.15.129:0/316550360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f10101ac0 0x7f8f10101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.962+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 >> 172.21.15.129:0/316550360 conn(0x7f8f100fc570 msgr2=0x7f8f10105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.963+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 shutdown_connections 2024-04-26T21:16:00.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:00.963+0000 7f8f14d97640 1 -- 172.21.15.129:0/316550360 wait complete. 2024-04-26T21:16:00.978 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:16:01.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.081+0000 7f5de9738640 1 -- 172.21.15.129:0/1343737782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4108660 msgr2=0x7f5de410aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.081+0000 7f5de9738640 1 --2- 172.21.15.129:0/1343737782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4108660 0x7f5de410aab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009ca0 tx=0x7f5ddc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.081+0000 7f5de9738640 1 -- 172.21.15.129:0/1343737782 shutdown_connections 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.081+0000 7f5de9738640 1 --2- 172.21.15.129:0/1343737782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5de4105d30 0x7f5de4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.081+0000 7f5de9738640 1 --2- 172.21.15.129:0/1343737782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4108660 0x7f5de410aab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.081+0000 7f5de9738640 1 --2- 172.21.15.129:0/1343737782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5de4103400 0x7f5de41057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.081+0000 7f5de9738640 1 -- 172.21.15.129:0/1343737782 >> 172.21.15.129:0/1343737782 conn(0x7f5de40fce10 msgr2=0x7f5de40ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.081+0000 7f5de9738640 1 -- 172.21.15.129:0/1343737782 shutdown_connections 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.081+0000 7f5de9738640 1 -- 172.21.15.129:0/1343737782 wait complete. 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de9738640 1 Processor -- start 2024-04-26T21:16:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de9738640 1 -- start start 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de9738640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4103400 0x7f5de4198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de9738640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5de4105d30 0x7f5de41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de9738640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5de4108660 0x7f5de41a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de9738640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5de410e4b0 con 0x7f5de4103400 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de9738640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5de410e330 con 0x7f5de4108660 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de9738640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5de410e630 con 0x7f5de4105d30 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4103400 0x7f5de4198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4103400 0x7f5de4198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45472/0 (socket says 172.21.15.129:45472) 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de2ffd640 1 -- 172.21.15.129:0/3107265723 learned_addr learned my addr 172.21.15.129:0/3107265723 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de37fe640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5de4108660 0x7f5de41a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de27fc640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5de4105d30 0x7f5de41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de2ffd640 1 -- 172.21.15.129:0/3107265723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5de4105d30 msgr2=0x7f5de41990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de2ffd640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5de4105d30 0x7f5de41990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de2ffd640 1 -- 172.21.15.129:0/3107265723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5de4108660 msgr2=0x7f5de41a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de2ffd640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5de4108660 0x7f5de41a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de2ffd640 1 -- 172.21.15.129:0/3107265723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de41a07a0 con 0x7f5de4103400 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.082+0000 7f5de37fe640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5de4108660 0x7f5de41a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.083+0000 7f5de2ffd640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4103400 0x7f5de4198b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d950 tx=0x7f5dd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.083+0000 7f5dcbfff640 1 -- 172.21.15.129:0/3107265723 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4017030 con 0x7f5de4103400 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.083+0000 7f5dcbfff640 1 -- 172.21.15.129:0/3107265723 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd4004020 con 0x7f5de4103400 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.083+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de41a0a90 con 0x7f5de4103400 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.083+0000 7f5dcbfff640 1 -- 172.21.15.129:0/3107265723 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40047a0 con 0x7f5de4103400 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.083+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41a0fe0 con 0x7f5de4103400 2024-04-26T21:16:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.084+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de40fe900 con 0x7f5de4103400 2024-04-26T21:16:01.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.085+0000 7f5dcbfff640 1 -- 172.21.15.129:0/3107265723 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5dd4009a30 con 0x7f5de4103400 2024-04-26T21:16:01.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.086+0000 7f5dcbfff640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5dcc0a3bb0 0x7f5dcc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.086+0000 7f5de27fc640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5dcc0a3bb0 0x7f5dcc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.086+0000 7f5dcbfff640 1 -- 172.21.15.129:0/3107265723 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5dd40cffe0 con 0x7f5de4103400 2024-04-26T21:16:01.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.086+0000 7f5de27fc640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5dcc0a3bb0 0x7f5dcc0a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5de419a090 tx=0x7f5dd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.089+0000 7f5dcbfff640 1 -- 172.21.15.129:0/3107265723 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5dd409ce60 con 0x7f5de4103400 2024-04-26T21:16:01.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.283+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5de402dc70 con 0x7f5de4103400 2024-04-26T21:16:01.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.284+0000 7f5dcbfff640 1 -- 172.21.15.129:0/3107265723 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f5dd409c760 con 0x7f5de4103400 2024-04-26T21:16:01.286 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:01.286 INFO:teuthology.orchestra.run.smithi129.stdout:[ 2024-04-26T21:16:01.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:16:01.286 INFO:teuthology.orchestra.run.smithi129.stdout: "poolnum": 1, 2024-04-26T21:16:01.286 INFO:teuthology.orchestra.run.smithi129.stdout: "poolname": ".mgr" 2024-04-26T21:16:01.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:16:01.286 INFO:teuthology.orchestra.run.smithi129.stdout:] 2024-04-26T21:16:01.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5dcc0a3bb0 msgr2=0x7f5dcc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5dcc0a3bb0 0x7f5dcc0a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5de419a090 tx=0x7f5dd8005f90 comp rx=0 tx=0).stop 2024-04-26T21:16:01.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4103400 msgr2=0x7f5de4198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4103400 0x7f5de4198b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d950 tx=0x7f5dd400de10 comp rx=0 tx=0).stop 2024-04-26T21:16:01.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 shutdown_connections 2024-04-26T21:16:01.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5dcc0a3bb0 0x7f5dcc0a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5de4108660 0x7f5de41a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5de4105d30 0x7f5de41990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 --2- 172.21.15.129:0/3107265723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5de4103400 0x7f5de4198b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 >> 172.21.15.129:0/3107265723 conn(0x7f5de40fce10 msgr2=0x7f5de4103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:01.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 shutdown_connections 2024-04-26T21:16:01.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.286+0000 7f5de9738640 1 -- 172.21.15.129:0/3107265723 wait complete. 2024-04-26T21:16:01.299 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:16:01.300 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:16:01.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 -- 172.21.15.129:0/3363030706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 msgr2=0x7f8934101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 --2- 172.21.15.129:0/3363030706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 0x7f8934101c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8928009900 tx=0x7f892802a410 comp rx=0 tx=0).stop 2024-04-26T21:16:01.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 -- 172.21.15.129:0/3363030706 shutdown_connections 2024-04-26T21:16:01.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 --2- 172.21.15.129:0/3363030706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8934102a60 0x7f8934102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 --2- 172.21.15.129:0/3363030706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8934103420 0x7f8934109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 --2- 172.21.15.129:0/3363030706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 0x7f8934101c90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 -- 172.21.15.129:0/3363030706 >> 172.21.15.129:0/3363030706 conn(0x7f89340fcde0 msgr2=0x7f89340ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 -- 172.21.15.129:0/3363030706 shutdown_connections 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 -- 172.21.15.129:0/3363030706 wait complete. 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.402+0000 7f893ac2a640 1 Processor -- start 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f893ac2a640 1 -- start start 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f893ac2a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 0x7f8934197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f893ac2a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8934102a60 0x7f8934198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f893ac2a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8934103420 0x7f893419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f893ac2a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f893410d3a0 con 0x7f8934101870 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f893ac2a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f893410d220 con 0x7f8934103420 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f893ac2a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f893410d520 con 0x7f8934102a60 2024-04-26T21:16:01.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89389c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 0x7f8934197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89389c6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 0x7f8934197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45482/0 (socket says 172.21.15.129:45482) 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89389c6640 1 -- 172.21.15.129:0/1915778970 learned_addr learned my addr 172.21.15.129:0/1915778970 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f8933fff640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8934102a60 0x7f8934198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89391c7640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8934103420 0x7f893419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89389c6640 1 -- 172.21.15.129:0/1915778970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8934102a60 msgr2=0x7f8934198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89389c6640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8934102a60 0x7f8934198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89389c6640 1 -- 172.21.15.129:0/1915778970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8934103420 msgr2=0x7f893419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89389c6640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8934103420 0x7f893419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89389c6640 1 -- 172.21.15.129:0/1915778970 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8934069ac0 con 0x7f8934101870 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f89389c6640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 0x7f8934197ad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8928006010 tx=0x7f89280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f8931ffb640 1 -- 172.21.15.129:0/1915778970 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892803f030 con 0x7f8934101870 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f8931ffb640 1 -- 172.21.15.129:0/1915778970 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8928031910 con 0x7f8934101870 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8934069d50 con 0x7f8934101870 2024-04-26T21:16:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.403+0000 7f8931ffb640 1 -- 172.21.15.129:0/1915778970 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928035070 con 0x7f8934101870 2024-04-26T21:16:01.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.404+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893406a240 con 0x7f8934101870 2024-04-26T21:16:01.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.405+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8934108c00 con 0x7f8934101870 2024-04-26T21:16:01.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.406+0000 7f8931ffb640 1 -- 172.21.15.129:0/1915778970 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8928042030 con 0x7f8934101870 2024-04-26T21:16:01.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.406+0000 7f8931ffb640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f89240a3bb0 0x7f89240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.406+0000 7f8931ffb640 1 -- 172.21.15.129:0/1915778970 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f89280f0500 con 0x7f8934101870 2024-04-26T21:16:01.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.406+0000 7f8933fff640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f89240a3bb0 0x7f89240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.407+0000 7f8933fff640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f89240a3bb0 0x7f89240a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8934198fe0 tx=0x7f8920005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.411+0000 7f8931ffb640 1 -- 172.21.15.129:0/1915778970 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f89280bd380 con 0x7f8934101870 2024-04-26T21:16:01.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:01.615+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f893402dad0 con 0x7f8934101870 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1408836150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/4252964298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/316550360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:01 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1408836150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/4252964298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/316550360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:01 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1408836150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:16:01.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/4252964298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:01.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/316550360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:16:01.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:01.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:01 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:02.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.248+0000 7f8931ffb640 1 -- 172.21.15.129:0/1915778970 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f89280bcc80 con 0x7f8934101870 2024-04-26T21:16:02.251 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_metadata' created 2024-04-26T21:16:02.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f89240a3bb0 msgr2=0x7f89240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f89240a3bb0 0x7f89240a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8934198fe0 tx=0x7f8920005f90 comp rx=0 tx=0).stop 2024-04-26T21:16:02.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 msgr2=0x7f8934197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 0x7f8934197ad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8928006010 tx=0x7f89280042f0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 shutdown_connections 2024-04-26T21:16:02.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f89240a3bb0 0x7f89240a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8934103420 0x7f893419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8934102a60 0x7f8934198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 --2- 172.21.15.129:0/1915778970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8934101870 0x7f8934197ad0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 >> 172.21.15.129:0/1915778970 conn(0x7f89340fcde0 msgr2=0x7f8934105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:02.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 shutdown_connections 2024-04-26T21:16:02.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.252+0000 7f893ac2a640 1 -- 172.21.15.129:0/1915778970 wait complete. 2024-04-26T21:16:02.268 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:16:02.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.382+0000 7f8ad987b640 1 -- 172.21.15.129:0/2775158311 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4100fe0 msgr2=0x7f8ad4101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.382+0000 7f8ad987b640 1 --2- 172.21.15.129:0/2775158311 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4100fe0 0x7f8ad4101400 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009900 tx=0x7f8ac402a410 comp rx=0 tx=0).stop 2024-04-26T21:16:02.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.382+0000 7f8ad987b640 1 -- 172.21.15.129:0/2775158311 shutdown_connections 2024-04-26T21:16:02.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.382+0000 7f8ad987b640 1 --2- 172.21.15.129:0/2775158311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad41021d0 0x7f8ad4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.382+0000 7f8ad987b640 1 --2- 172.21.15.129:0/2775158311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ad4102b90 0x7f8ad4109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.382+0000 7f8ad987b640 1 --2- 172.21.15.129:0/2775158311 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4100fe0 0x7f8ad4101400 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.382+0000 7f8ad987b640 1 -- 172.21.15.129:0/2775158311 >> 172.21.15.129:0/2775158311 conn(0x7f8ad40fc550 msgr2=0x7f8ad40fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:02.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.382+0000 7f8ad987b640 1 -- 172.21.15.129:0/2775158311 shutdown_connections 2024-04-26T21:16:02.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.382+0000 7f8ad987b640 1 -- 172.21.15.129:0/2775158311 wait complete. 2024-04-26T21:16:02.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad987b640 1 Processor -- start 2024-04-26T21:16:02.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad987b640 1 -- start start 2024-04-26T21:16:02.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad987b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ad4100fe0 0x7f8ad4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad987b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad41021d0 0x7f8ad4196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad987b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4102b90 0x7f8ad4196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad987b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410c2f0 con 0x7f8ad4102b90 2024-04-26T21:16:02.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad987b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410c170 con 0x7f8ad41021d0 2024-04-26T21:16:02.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad987b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410c470 con 0x7f8ad4100fe0 2024-04-26T21:16:02.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4102b90 0x7f8ad4196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4102b90 0x7f8ad4196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45500/0 (socket says 172.21.15.129:45500) 2024-04-26T21:16:02.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ad4100fe0 0x7f8ad4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad37fe640 1 -- 172.21.15.129:0/3158419409 learned_addr learned my addr 172.21.15.129:0/3158419409 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:02.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.383+0000 7f8ad27fc640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad41021d0 0x7f8ad4196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad37fe640 1 -- 172.21.15.129:0/3158419409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ad4100fe0 msgr2=0x7f8ad4198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad37fe640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ad4100fe0 0x7f8ad4198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad37fe640 1 -- 172.21.15.129:0/3158419409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad41021d0 msgr2=0x7f8ad4196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad37fe640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad41021d0 0x7f8ad4196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad37fe640 1 -- 172.21.15.129:0/3158419409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad4197520 con 0x7f8ad4102b90 2024-04-26T21:16:02.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad2ffd640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ad4100fe0 0x7f8ad4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:16:02.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad37fe640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4102b90 0x7f8ad4196c90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00eb10 tx=0x7f8acc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:02.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad8879640 1 -- 172.21.15.129:0/3158419409 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc004b20 con 0x7f8ad4102b90 2024-04-26T21:16:02.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad41a2cc0 con 0x7f8ad4102b90 2024-04-26T21:16:02.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad8879640 1 -- 172.21.15.129:0/3158419409 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8acc004cc0 con 0x7f8ad4102b90 2024-04-26T21:16:02.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad8879640 1 -- 172.21.15.129:0/3158419409 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc010690 con 0x7f8ad4102b90 2024-04-26T21:16:02.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.384+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad41a31b0 con 0x7f8ad4102b90 2024-04-26T21:16:02.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.385+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad4108300 con 0x7f8ad4102b90 2024-04-26T21:16:02.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.386+0000 7f8ad8879640 1 -- 172.21.15.129:0/3158419409 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8acc005000 con 0x7f8ad4102b90 2024-04-26T21:16:02.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.387+0000 7f8ad8879640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8abc0a3bb0 0x7f8abc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.387+0000 7f8ad2ffd640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8abc0a3bb0 0x7f8abc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.387+0000 7f8ad8879640 1 -- 172.21.15.129:0/3158419409 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f8acc0d0880 con 0x7f8ad4102b90 2024-04-26T21:16:02.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.388+0000 7f8ad2ffd640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8abc0a3bb0 0x7f8abc0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4005ab0 tx=0x7f8ac4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:02.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.391+0000 7f8ad8879640 1 -- 172.21.15.129:0/3158419409 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8acc054060 con 0x7f8ad4102b90 2024-04-26T21:16:02.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:02.587+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8ad402dad0 con 0x7f8ad4102b90 2024-04-26T21:16:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:02 smithi150 ceph-mon[32290]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:02 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3107265723' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:16:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:02 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1915778970' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:16:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:02 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:02 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:02 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:02 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:02.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:02 smithi171 ceph-mon[33478]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:02.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:02 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3107265723' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:16:02.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:02 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1915778970' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:16:02.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:02 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:02.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:02 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:02.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:02 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:02.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:02 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:02.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:02 smithi129 ceph-mon[29237]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:02.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:02 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3107265723' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:16:02.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:02 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1915778970' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:16:02.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:02 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:02.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:02 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:02.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:02 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:02.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:02 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:03.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.256+0000 7f8ad8879640 1 -- 172.21.15.129:0/3158419409 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f8acc09d590 con 0x7f8ad4102b90 2024-04-26T21:16:03.258 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data' created 2024-04-26T21:16:03.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8abc0a3bb0 msgr2=0x7f8abc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8abc0a3bb0 0x7f8abc0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4005ab0 tx=0x7f8ac4005a20 comp rx=0 tx=0).stop 2024-04-26T21:16:03.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4102b90 msgr2=0x7f8ad4196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4102b90 0x7f8ad4196c90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00eb10 tx=0x7f8acc00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 shutdown_connections 2024-04-26T21:16:03.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8abc0a3bb0 0x7f8abc0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad41021d0 0x7f8ad4196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ad4102b90 0x7f8ad4196c90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 --2- 172.21.15.129:0/3158419409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ad4100fe0 0x7f8ad4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 >> 172.21.15.129:0/3158419409 conn(0x7f8ad40fc550 msgr2=0x7f8ad41054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 shutdown_connections 2024-04-26T21:16:03.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.259+0000 7f8ad987b640 1 -- 172.21.15.129:0/3158419409 wait complete. 2024-04-26T21:16:03.288 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:16:03.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.406+0000 7fc12a114640 1 -- 172.21.15.129:0/719584927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124102e40 msgr2=0x7fc124103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.407+0000 7fc12a114640 1 --2- 172.21.15.129:0/719584927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124102e40 0x7fc124103260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc118009860 tx=0x7fc11802a340 comp rx=0 tx=0).stop 2024-04-26T21:16:03.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.407+0000 7fc12a114640 1 -- 172.21.15.129:0/719584927 shutdown_connections 2024-04-26T21:16:03.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.407+0000 7fc12a114640 1 --2- 172.21.15.129:0/719584927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124101a90 0x7fc124101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.407+0000 7fc12a114640 1 --2- 172.21.15.129:0/719584927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc124102450 0x7fc124109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.407+0000 7fc12a114640 1 --2- 172.21.15.129:0/719584927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124102e40 0x7fc124103260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.407+0000 7fc12a114640 1 -- 172.21.15.129:0/719584927 >> 172.21.15.129:0/719584927 conn(0x7fc1240fc560 msgr2=0x7fc1240fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.407+0000 7fc12a114640 1 -- 172.21.15.129:0/719584927 shutdown_connections 2024-04-26T21:16:03.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.407+0000 7fc12a114640 1 -- 172.21.15.129:0/719584927 wait complete. 2024-04-26T21:16:03.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.407+0000 7fc12a114640 1 Processor -- start 2024-04-26T21:16:03.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc12a114640 1 -- start start 2024-04-26T21:16:03.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc12a114640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc124101a90 0x7fc1241014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc12a114640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124102450 0x7fc12419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc12a114640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124102e40 0x7fc12419ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc12a114640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc12410d380 con 0x7fc124101a90 2024-04-26T21:16:03.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc12a114640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc12410d200 con 0x7fc124102450 2024-04-26T21:16:03.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc12a114640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc12410d500 con 0x7fc124102e40 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc1237fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc124101a90 0x7fc1241014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc122ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124102450 0x7fc12419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc123fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124102e40 0x7fc12419ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc1237fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc124101a90 0x7fc1241014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33272/0 (socket says 172.21.15.129:33272) 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc1237fe640 1 -- 172.21.15.129:0/646104650 learned_addr learned my addr 172.21.15.129:0/646104650 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc122ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124102450 0x7fc12419d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:39310/0 (socket says 172.21.15.129:39310) 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc123fff640 1 -- 172.21.15.129:0/646104650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124102450 msgr2=0x7fc12419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc123fff640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124102450 0x7fc12419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc123fff640 1 -- 172.21.15.129:0/646104650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc124101a90 msgr2=0x7fc1241014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc123fff640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc124101a90 0x7fc1241014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.408+0000 7fc123fff640 1 -- 172.21.15.129:0/646104650 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1241a0890 con 0x7fc124102e40 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.409+0000 7fc123fff640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124102e40 0x7fc12419ff80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00ef00 tx=0x7fc11c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.409+0000 7fc120ff9640 1 -- 172.21.15.129:0/646104650 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c004b20 con 0x7fc124102e40 2024-04-26T21:16:03.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.409+0000 7fc120ff9640 1 -- 172.21.15.129:0/646104650 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc11c003e20 con 0x7fc124102e40 2024-04-26T21:16:03.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.409+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1241a0b20 con 0x7fc124102e40 2024-04-26T21:16:03.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.409+0000 7fc120ff9640 1 -- 172.21.15.129:0/646104650 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c00b570 con 0x7fc124102e40 2024-04-26T21:16:03.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.409+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1241a1070 con 0x7fc124102e40 2024-04-26T21:16:03.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.410+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc124108d40 con 0x7fc124102e40 2024-04-26T21:16:03.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.411+0000 7fc120ff9640 1 -- 172.21.15.129:0/646104650 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc11c01d020 con 0x7fc124102e40 2024-04-26T21:16:03.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.412+0000 7fc120ff9640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc10c0a3c30 0x7fc10c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.412+0000 7fc1237fe640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc10c0a3c30 0x7fc10c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.412+0000 7fc120ff9640 1 -- 172.21.15.129:0/646104650 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fc11c0d0350 con 0x7fc124102e40 2024-04-26T21:16:03.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.412+0000 7fc1237fe640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc10c0a3c30 0x7fc10c0a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc1180095f0 tx=0x7fc1180058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.417+0000 7fc120ff9640 1 -- 172.21.15.129:0/646104650 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc11c004cc0 con 0x7fc124102e40 2024-04-26T21:16:03.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:03 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1915778970' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:16:03.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:03 smithi150 ceph-mon[32290]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:16:03.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:03 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3158419409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:16:03.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:03.675+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc124063640 con 0x7fc124102e40 2024-04-26T21:16:03.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:03 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1915778970' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:16:03.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:03 smithi129 ceph-mon[29237]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:16:03.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:03 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3158419409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:16:03.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:03 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1915778970' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:16:03.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:03 smithi171 ceph-mon[33478]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:16:03.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:03 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3158419409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3158419409' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: pgmap v288: 97 pgs: 10 creating+peering, 86 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:04.623 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/646104650' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:16:04.623 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:16:04.623 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:16:04.623 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:04.623 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.623 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:04 smithi129 ceph-mon[29237]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3158419409' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: pgmap v288: 97 pgs: 10 creating+peering, 86 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/646104650' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:16:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:16:04.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:16:04.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:04.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:04 smithi150 ceph-mon[32290]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:16:04.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.677+0000 7fc120ff9640 1 -- 172.21.15.129:0/646104650 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc11c09ce40 con 0x7fc124102e40 2024-04-26T21:16:04.680 INFO:teuthology.orchestra.run.smithi129.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:16:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc10c0a3c30 msgr2=0x7fc10c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc10c0a3c30 0x7fc10c0a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc1180095f0 tx=0x7fc1180058b0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124102e40 msgr2=0x7fc12419ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124102e40 0x7fc12419ff80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00ef00 tx=0x7fc11c00c740 comp rx=0 tx=0).stop 2024-04-26T21:16:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 shutdown_connections 2024-04-26T21:16:04.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc10c0a3c30 0x7fc10c0a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124102450 0x7fc12419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124102e40 0x7fc12419ff80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 --2- 172.21.15.129:0/646104650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc124101a90 0x7fc1241014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.680+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 >> 172.21.15.129:0/646104650 conn(0x7fc1240fc560 msgr2=0x7fc124105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:04.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.681+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 shutdown_connections 2024-04-26T21:16:04.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.681+0000 7fc12a114640 1 -- 172.21.15.129:0/646104650 wait complete. 2024-04-26T21:16:04.700 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3158419409' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: pgmap v288: 97 pgs: 10 creating+peering, 86 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:04.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/646104650' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:16:04.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:16:04.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:16:04.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:04.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:16:04.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:04 smithi171 ceph-mon[33478]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:16:04.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 -- 172.21.15.129:0/3550124723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 msgr2=0x7fac94101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 --2- 172.21.15.129:0/3550124723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 0x7fac94101c90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009860 tx=0x7fac8c02a340 comp rx=0 tx=0).stop 2024-04-26T21:16:04.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 -- 172.21.15.129:0/3550124723 shutdown_connections 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 --2- 172.21.15.129:0/3550124723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac94103420 0x7fac94109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 --2- 172.21.15.129:0/3550124723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac94102a60 0x7fac94102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 --2- 172.21.15.129:0/3550124723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 0x7fac94101c90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 -- 172.21.15.129:0/3550124723 >> 172.21.15.129:0/3550124723 conn(0x7fac940fcde0 msgr2=0x7fac940ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 -- 172.21.15.129:0/3550124723 shutdown_connections 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 -- 172.21.15.129:0/3550124723 wait complete. 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.809+0000 7fac995c9640 1 Processor -- start 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac995c9640 1 -- start start 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac995c9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 0x7fac94197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac995c9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac94102a60 0x7fac94198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac995c9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac94103420 0x7fac9419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac995c9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac9410d370 con 0x7fac94101870 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac995c9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac9410d1f0 con 0x7fac94102a60 2024-04-26T21:16:04.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac995c9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fac9410d4f0 con 0x7fac94103420 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac92ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 0x7fac94197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac92ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 0x7fac94197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33296/0 (socket says 172.21.15.129:33296) 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac92ffd640 1 -- 172.21.15.129:0/4054775819 learned_addr learned my addr 172.21.15.129:0/4054775819 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac927fc640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac94102a60 0x7fac94198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac92ffd640 1 -- 172.21.15.129:0/4054775819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac94103420 msgr2=0x7fac9419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac92ffd640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac94103420 0x7fac9419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac92ffd640 1 -- 172.21.15.129:0/4054775819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac94102a60 msgr2=0x7fac94198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac92ffd640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac94102a60 0x7fac94198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac92ffd640 1 -- 172.21.15.129:0/4054775819 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9419f710 con 0x7fac94101870 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac92ffd640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 0x7fac94197af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac8c005940 tx=0x7fac8c032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac7bfff640 1 -- 172.21.15.129:0/4054775819 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c040020 con 0x7fac94101870 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.810+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9419f9a0 con 0x7fac94101870 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.811+0000 7fac7bfff640 1 -- 172.21.15.129:0/4054775819 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac8c004040 con 0x7fac94101870 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.811+0000 7fac7bfff640 1 -- 172.21.15.129:0/4054775819 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c037040 con 0x7fac94101870 2024-04-26T21:16:04.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.811+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9419fee0 con 0x7fac94101870 2024-04-26T21:16:04.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.811+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac94108c50 con 0x7fac94101870 2024-04-26T21:16:04.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.813+0000 7fac7bfff640 1 -- 172.21.15.129:0/4054775819 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fac8c043020 con 0x7fac94101870 2024-04-26T21:16:04.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.813+0000 7fac7bfff640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac7c0a3bb0 0x7fac7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:04.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.813+0000 7fac927fc640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac7c0a3bb0 0x7fac7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:04.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.813+0000 7fac7bfff640 1 -- 172.21.15.129:0/4054775819 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fac8c0f0920 con 0x7fac94101870 2024-04-26T21:16:04.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.814+0000 7fac927fc640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac7c0a3bb0 0x7fac7c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fac880096c0 tx=0x7fac88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:04.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:04.817+0000 7fac7bfff640 1 -- 172.21.15.129:0/4054775819 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fac8c04a020 con 0x7fac94101870 2024-04-26T21:16:05.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.000+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac94063530 con 0x7fac94101870 2024-04-26T21:16:05.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.002+0000 7fac7bfff640 1 -- 172.21.15.129:0/4054775819 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22274 (secure 0 0 0) 0x7fac8c0bd4d0 con 0x7fac94101870 2024-04-26T21:16:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:05.004 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:04.683012+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:16:05.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.004+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac7c0a3bb0 msgr2=0x7fac7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.004+0000 7fac995c9640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac7c0a3bb0 0x7fac7c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fac880096c0 tx=0x7fac88009260 comp rx=0 tx=0).stop 2024-04-26T21:16:05.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.004+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 msgr2=0x7fac94197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.004+0000 7fac995c9640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 0x7fac94197af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac8c005940 tx=0x7fac8c032dc0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.005+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 shutdown_connections 2024-04-26T21:16:05.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.005+0000 7fac995c9640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fac7c0a3bb0 0x7fac7c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.005+0000 7fac995c9640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac94102a60 0x7fac94198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.005+0000 7fac995c9640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac94103420 0x7fac9419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.005+0000 7fac995c9640 1 --2- 172.21.15.129:0/4054775819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac94101870 0x7fac94197af0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.005+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 >> 172.21.15.129:0/4054775819 conn(0x7fac940fcde0 msgr2=0x7fac94105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:05.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.005+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 shutdown_connections 2024-04-26T21:16:05.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.005+0000 7fac995c9640 1 -- 172.21.15.129:0/4054775819 wait complete. 2024-04-26T21:16:05.018 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.115+0000 7f11a995f640 1 -- 172.21.15.129:0/2243057317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4102980 msgr2=0x7f11a4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.115+0000 7f11a995f640 1 --2- 172.21.15.129:0/2243057317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4102980 0x7f11a4102da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1194009900 tx=0x7f119402a410 comp rx=0 tx=0).stop 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.115+0000 7f11a995f640 1 -- 172.21.15.129:0/2243057317 shutdown_connections 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.115+0000 7f11a995f640 1 --2- 172.21.15.129:0/2243057317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a4104530 0x7f11a410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.115+0000 7f11a995f640 1 --2- 172.21.15.129:0/2243057317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a4103b70 0x7f11a4103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.115+0000 7f11a995f640 1 --2- 172.21.15.129:0/2243057317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4102980 0x7f11a4102da0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.115+0000 7f11a995f640 1 -- 172.21.15.129:0/2243057317 >> 172.21.15.129:0/2243057317 conn(0x7f11a40fdf10 msgr2=0x7f11a4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 -- 172.21.15.129:0/2243057317 shutdown_connections 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 -- 172.21.15.129:0/2243057317 wait complete. 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 Processor -- start 2024-04-26T21:16:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 -- start start 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a4102980 0x7f11a4198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a4103b70 0x7f11a4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4104530 0x7f11a41a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11a410e4a0 con 0x7f11a4104530 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11a410e320 con 0x7f11a4102980 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a995f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f11a410e620 con 0x7f11a4103b70 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.117+0000 7f11a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4104530 0x7f11a41a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4104530 0x7f11a41a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33326/0 (socket says 172.21.15.129:33326) 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a37fe640 1 -- 172.21.15.129:0/2456149536 learned_addr learned my addr 172.21.15.129:0/2456149536 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:05.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a27fc640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a4103b70 0x7f11a4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a2ffd640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a4102980 0x7f11a4198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a37fe640 1 -- 172.21.15.129:0/2456149536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a4103b70 msgr2=0x7f11a4199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a37fe640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a4103b70 0x7f11a4199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a37fe640 1 -- 172.21.15.129:0/2456149536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a4102980 msgr2=0x7f11a4198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a37fe640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a4102980 0x7f11a4198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a37fe640 1 -- 172.21.15.129:0/2456149536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a41a0840 con 0x7f11a4104530 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a2ffd640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a4102980 0x7f11a4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a37fe640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4104530 0x7f11a41a01a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f119c00cad0 tx=0x7f119c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a895d640 1 -- 172.21.15.129:0/2456149536 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c00ae20 con 0x7f11a4104530 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a895d640 1 -- 172.21.15.129:0/2456149536 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f119c005020 con 0x7f11a4104530 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a41a0b30 con 0x7f11a4104530 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a895d640 1 -- 172.21.15.129:0/2456149536 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c00f6b0 con 0x7f11a4104530 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.118+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a41a1030 con 0x7f11a4104530 2024-04-26T21:16:05.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.119+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a4109d10 con 0x7f11a4104530 2024-04-26T21:16:05.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.120+0000 7f11a895d640 1 -- 172.21.15.129:0/2456149536 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f119c0051d0 con 0x7f11a4104530 2024-04-26T21:16:05.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.121+0000 7f11a895d640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f118c0a3bb0 0x7f118c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.121+0000 7f11a895d640 1 -- 172.21.15.129:0/2456149536 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f119c0d07b0 con 0x7f11a4104530 2024-04-26T21:16:05.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.121+0000 7f11a2ffd640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f118c0a3bb0 0x7f118c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.121+0000 7f11a2ffd640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f118c0a3bb0 0x7f118c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1194009230 tx=0x7f11940314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.129+0000 7f11a895d640 1 -- 172.21.15.129:0/2456149536 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f119c014030 con 0x7f11a4104530 2024-04-26T21:16:05.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.316+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11a402dd20 con 0x7f11a4104530 2024-04-26T21:16:05.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.318+0000 7f11a895d640 1 -- 172.21.15.129:0/2456149536 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22274 (secure 0 0 0) 0x7f119c09d2c0 con 0x7f11a4104530 2024-04-26T21:16:05.319 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:05.319 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:04.683012+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:16:05.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f118c0a3bb0 msgr2=0x7f118c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f118c0a3bb0 0x7f118c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1194009230 tx=0x7f11940314a0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4104530 msgr2=0x7f11a41a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4104530 0x7f11a41a01a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f119c00cad0 tx=0x7f119c00cf90 comp rx=0 tx=0).stop 2024-04-26T21:16:05.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 shutdown_connections 2024-04-26T21:16:05.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f118c0a3bb0 0x7f118c0a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11a4104530 0x7f11a41a01a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a4103b70 0x7f11a4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 --2- 172.21.15.129:0/2456149536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a4102980 0x7f11a4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 >> 172.21.15.129:0/2456149536 conn(0x7f11a40fdf10 msgr2=0x7f11a4106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:05.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 shutdown_connections 2024-04-26T21:16:05.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.320+0000 7f11a995f640 1 -- 172.21.15.129:0/2456149536 wait complete. 2024-04-26T21:16:05.331 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:05 smithi129 ceph-mon[29237]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:16:05.331 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:05 smithi129 ceph-mon[29237]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:16:05.331 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:05 smithi129 ceph-mon[29237]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:16:05.331 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:05 smithi129 ceph-mon[29237]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:16:05.331 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:05 smithi129 ceph-mon[29237]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:16:05.331 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:05 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/4054775819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:05.333 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.434+0000 7f37bdeb0640 1 -- 172.21.15.129:0/1478437764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8103420 msgr2=0x7f37b8109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.434+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/1478437764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8103420 0x7f37b8109cb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009ca0 tx=0x7f37b002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.434+0000 7f37bdeb0640 1 -- 172.21.15.129:0/1478437764 shutdown_connections 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.434+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/1478437764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b8102a60 0x7f37b8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.434+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/1478437764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8103420 0x7f37b8109cb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.434+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/1478437764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b8101870 0x7f37b8101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.434+0000 7f37bdeb0640 1 -- 172.21.15.129:0/1478437764 >> 172.21.15.129:0/1478437764 conn(0x7f37b80fcde0 msgr2=0x7f37b80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.434+0000 7f37bdeb0640 1 -- 172.21.15.129:0/1478437764 shutdown_connections 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37bdeb0640 1 -- 172.21.15.129:0/1478437764 wait complete. 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37bdeb0640 1 Processor -- start 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37bdeb0640 1 -- start start 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37bdeb0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8101870 0x7f37b806a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37bdeb0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b8102a60 0x7f37b819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37bdeb0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b8103420 0x7f37b819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37bdeb0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f37b810d3a0 con 0x7f37b8101870 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37bdeb0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f37b810d220 con 0x7f37b8103420 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37bdeb0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f37b810d520 con 0x7f37b8102a60 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8101870 0x7f37b806a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8101870 0x7f37b806a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33338/0 (socket says 172.21.15.129:33338) 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.435+0000 7f37b77fe640 1 -- 172.21.15.129:0/3218463319 learned_addr learned my addr 172.21.15.129:0/3218463319 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b7fff640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b8103420 0x7f37b819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b77fe640 1 -- 172.21.15.129:0/3218463319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b8102a60 msgr2=0x7f37b819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b6ffd640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b8102a60 0x7f37b819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b77fe640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b8102a60 0x7f37b819d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b77fe640 1 -- 172.21.15.129:0/3218463319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b8103420 msgr2=0x7f37b819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b77fe640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b8103420 0x7f37b819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b77fe640 1 -- 172.21.15.129:0/3218463319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b81a0790 con 0x7f37b8101870 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b7fff640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b8103420 0x7f37b819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b77fe640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8101870 0x7f37b806a630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00e9c0 tx=0x7f37ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b4ff9640 1 -- 172.21.15.129:0/3218463319 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00cd80 con 0x7f37b8101870 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b4ff9640 1 -- 172.21.15.129:0/3218463319 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37ac0035f0 con 0x7f37b8101870 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b81a0a20 con 0x7f37b8101870 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37b4ff9640 1 -- 172.21.15.129:0/3218463319 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac010690 con 0x7f37b8101870 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.436+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81a0f20 con 0x7f37b8101870 2024-04-26T21:16:05.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.437+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b8108b90 con 0x7f37b8101870 2024-04-26T21:16:05.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.438+0000 7f37b4ff9640 1 -- 172.21.15.129:0/3218463319 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f37ac004000 con 0x7f37b8101870 2024-04-26T21:16:05.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.439+0000 7f37b4ff9640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37a00a3bb0 0x7f37a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.439+0000 7f37b6ffd640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37a00a3bb0 0x7f37a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.439+0000 7f37b4ff9640 1 -- 172.21.15.129:0/3218463319 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f37ac0d0a90 con 0x7f37b8101870 2024-04-26T21:16:05.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.439+0000 7f37b6ffd640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37a00a3bb0 0x7f37a00a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f37b806a3a0 tx=0x7f37a8005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.446+0000 7f37b4ff9640 1 -- 172.21.15.129:0/3218463319 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f37ac09d5c0 con 0x7f37b8101870 2024-04-26T21:16:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:05 smithi150 ceph-mon[32290]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:16:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:05 smithi150 ceph-mon[32290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:16:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:05 smithi150 ceph-mon[32290]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:16:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:05 smithi150 ceph-mon[32290]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:16:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:05 smithi150 ceph-mon[32290]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:16:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:05 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/4054775819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:05.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:05.668+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f37b802dc90 con 0x7f37b8101870 2024-04-26T21:16:05.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:05 smithi171 ceph-mon[33478]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:16:05.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:05 smithi171 ceph-mon[33478]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:16:05.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:05 smithi171 ceph-mon[33478]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:16:05.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:05 smithi171 ceph-mon[33478]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:16:05.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:05 smithi171 ceph-mon[33478]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:16:05.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:05 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/4054775819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:06 smithi150 ceph-mon[32290]: pgmap v291: 97 pgs: 14 creating+peering, 61 unknown, 22 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:06 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2456149536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:06 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] up:active 2024-04-26T21:16:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:06 smithi150 ceph-mon[32290]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:16:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:06 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3218463319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:16:06.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.666+0000 7f37b4ff9640 1 -- 172.21.15.129:0/3218463319 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f37ac09cec0 con 0x7f37b8101870 2024-04-26T21:16:06.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37a00a3bb0 msgr2=0x7f37a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37a00a3bb0 0x7f37a00a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f37b806a3a0 tx=0x7f37a8005cc0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8101870 msgr2=0x7f37b806a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8101870 0x7f37b806a630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00e9c0 tx=0x7f37ac00ee80 comp rx=0 tx=0).stop 2024-04-26T21:16:06.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 shutdown_connections 2024-04-26T21:16:06.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37a00a3bb0 0x7f37a00a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b8103420 0x7f37b819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b8102a60 0x7f37b819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 --2- 172.21.15.129:0/3218463319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37b8101870 0x7f37b806a630 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 >> 172.21.15.129:0/3218463319 conn(0x7f37b80fcde0 msgr2=0x7f37b8105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:06.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 shutdown_connections 2024-04-26T21:16:06.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.669+0000 7f37bdeb0640 1 -- 172.21.15.129:0/3218463319 wait complete. 2024-04-26T21:16:06.682 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:06 smithi129 ceph-mon[29237]: pgmap v291: 97 pgs: 14 creating+peering, 61 unknown, 22 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:06.682 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:06 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2456149536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:06.682 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:06 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] up:active 2024-04-26T21:16:06.682 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:06 smithi129 ceph-mon[29237]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:16:06.682 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:06 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3218463319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:16:06.683 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-26T21:16:06.683 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-26T21:16:06.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:06 smithi171 ceph-mon[33478]: pgmap v291: 97 pgs: 14 creating+peering, 61 unknown, 22 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:16:06.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:06 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2456149536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:06.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:06 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] up:active 2024-04-26T21:16:06.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:06 smithi171 ceph-mon[33478]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:16:06.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:06 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3218463319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:16:06.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.788+0000 7f756a293640 1 -- 172.21.15.129:0/2735639597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564102980 msgr2=0x7f7564102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.788+0000 7f756a293640 1 --2- 172.21.15.129:0/2735639597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564102980 0x7f7564102da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7554009900 tx=0x7f755402a410 comp rx=0 tx=0).stop 2024-04-26T21:16:06.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.788+0000 7f756a293640 1 -- 172.21.15.129:0/2735639597 shutdown_connections 2024-04-26T21:16:06.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.788+0000 7f756a293640 1 --2- 172.21.15.129:0/2735639597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7564103b70 0x7f7564103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.788+0000 7f756a293640 1 --2- 172.21.15.129:0/2735639597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564104530 0x7f756410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.788+0000 7f756a293640 1 --2- 172.21.15.129:0/2735639597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564102980 0x7f7564102da0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.788+0000 7f756a293640 1 -- 172.21.15.129:0/2735639597 >> 172.21.15.129:0/2735639597 conn(0x7f75640fdf10 msgr2=0x7f7564100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.788+0000 7f756a293640 1 -- 172.21.15.129:0/2735639597 shutdown_connections 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.788+0000 7f756a293640 1 -- 172.21.15.129:0/2735639597 wait complete. 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.789+0000 7f756a293640 1 Processor -- start 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.789+0000 7f756a293640 1 -- start start 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.789+0000 7f756a293640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7564102980 0x7f756419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.789+0000 7f756a293640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564103b70 0x7f756419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.789+0000 7f756a293640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564104530 0x7f75641a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.789+0000 7f756a293640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f756410e500 con 0x7f7564104530 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.789+0000 7f756a293640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f756410e380 con 0x7f7564102980 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.789+0000 7f756a293640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f756410e680 con 0x7f7564103b70 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.789+0000 7f7568830640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564104530 0x7f75641a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f7568830640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564104530 0x7f75641a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33360/0 (socket says 172.21.15.129:33360) 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f7568830640 1 -- 172.21.15.129:0/3841885426 learned_addr learned my addr 172.21.15.129:0/3841885426 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f7563fff640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7564102980 0x7f756419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f75637fe640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564103b70 0x7f756419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f7568830640 1 -- 172.21.15.129:0/3841885426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564103b70 msgr2=0x7f756419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f7568830640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564103b70 0x7f756419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f7568830640 1 -- 172.21.15.129:0/3841885426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7564102980 msgr2=0x7f756419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f7568830640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7564102980 0x7f756419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f7568830640 1 -- 172.21.15.129:0/3841885426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75641a2a60 con 0x7f7564104530 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f75637fe640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564103b70 0x7f756419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f7568830640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564104530 0x7f75641a23f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f755c00b970 tx=0x7f755c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f75617fa640 1 -- 172.21.15.129:0/3841885426 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c019070 con 0x7f7564104530 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f75617fa640 1 -- 172.21.15.129:0/3841885426 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755c014440 con 0x7f7564104530 2024-04-26T21:16:06.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f75617fa640 1 -- 172.21.15.129:0/3841885426 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c0075d0 con 0x7f7564104530 2024-04-26T21:16:06.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75641a2d50 con 0x7f7564104530 2024-04-26T21:16:06.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.790+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75641a32c0 con 0x7f7564104530 2024-04-26T21:16:06.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.791+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75641a3550 con 0x7f7564104530 2024-04-26T21:16:06.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.792+0000 7f75617fa640 1 -- 172.21.15.129:0/3841885426 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f755c01c070 con 0x7f7564104530 2024-04-26T21:16:06.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.793+0000 7f75617fa640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f754c0a3c30 0x7f754c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.793+0000 7f7563fff640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f754c0a3c30 0x7f754c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.793+0000 7f75617fa640 1 -- 172.21.15.129:0/3841885426 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f755c0cf7d0 con 0x7f7564104530 2024-04-26T21:16:06.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.793+0000 7f7563fff640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f754c0a3c30 0x7f754c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7554009230 tx=0x7f75540314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:06.797+0000 7f75617fa640 1 -- 172.21.15.129:0/3841885426 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f755c09bcb0 con 0x7f7564104530 2024-04-26T21:16:07.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.013+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f75641a3940 con 0x7f7564104530 2024-04-26T21:16:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:07 smithi150 ceph-mon[32290]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T21:16:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:07 smithi150 ceph-mon[32290]: Cluster is now healthy 2024-04-26T21:16:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:07 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3218463319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:16:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:07 smithi150 ceph-mon[32290]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:16:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:07 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3841885426' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:16:07.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.667+0000 7f75617fa640 1 -- 172.21.15.129:0/3841885426 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f755c09ffb0 con 0x7f7564104530 2024-04-26T21:16:07.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f754c0a3c30 msgr2=0x7f754c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f754c0a3c30 0x7f754c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7554009230 tx=0x7f75540314a0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564104530 msgr2=0x7f75641a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564104530 0x7f75641a23f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f755c00b970 tx=0x7f755c00be30 comp rx=0 tx=0).stop 2024-04-26T21:16:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 shutdown_connections 2024-04-26T21:16:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f754c0a3c30 0x7f754c0a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564104530 0x7f75641a23f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564103b70 0x7f756419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 --2- 172.21.15.129:0/3841885426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7564102980 0x7f756419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 >> 172.21.15.129:0/3841885426 conn(0x7f75640fdf10 msgr2=0x7f7564109390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 shutdown_connections 2024-04-26T21:16:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.670+0000 7f756a293640 1 -- 172.21.15.129:0/3841885426 wait complete. 2024-04-26T21:16:07.684 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:07 smithi129 ceph-mon[29237]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T21:16:07.684 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:07 smithi129 ceph-mon[29237]: Cluster is now healthy 2024-04-26T21:16:07.684 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:07 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3218463319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:16:07.684 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:07 smithi129 ceph-mon[29237]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:16:07.684 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:07 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3841885426' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:16:07.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-26T21:16:07.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:07 smithi171 ceph-mon[33478]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T21:16:07.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:07 smithi171 ceph-mon[33478]: Cluster is now healthy 2024-04-26T21:16:07.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:07 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3218463319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:16:07.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:07 smithi171 ceph-mon[33478]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:16:07.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:07 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3841885426' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:16:07.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 -- 172.21.15.129:0/186364540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4103440 msgr2=0x7fcbf4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 --2- 172.21.15.129:0/186364540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4103440 0x7fcbf4109cd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009ca0 tx=0x7fcbec02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 -- 172.21.15.129:0/186364540 shutdown_connections 2024-04-26T21:16:07.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 --2- 172.21.15.129:0/186364540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4103440 0x7fcbf4109cd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 --2- 172.21.15.129:0/186364540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbf4102a80 0x7fcbf4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 --2- 172.21.15.129:0/186364540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf4101890 0x7fcbf4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 -- 172.21.15.129:0/186364540 >> 172.21.15.129:0/186364540 conn(0x7fcbf40fce20 msgr2=0x7fcbf40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 -- 172.21.15.129:0/186364540 shutdown_connections 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 -- 172.21.15.129:0/186364540 wait complete. 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.790+0000 7fcbfa346640 1 Processor -- start 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbfa346640 1 -- start start 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbfa346640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4101890 0x7fcbf4105820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbfa346640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbf4102a80 0x7fcbf4103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbfa346640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf4103440 0x7fcbf41043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbfa346640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf410d3c0 con 0x7fcbf4101890 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbfa346640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf410d240 con 0x7fcbf4103440 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbfa346640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf410d540 con 0x7fcbf4102a80 2024-04-26T21:16:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4101890 0x7fcbf4105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4101890 0x7fcbf4105820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33388/0 (socket says 172.21.15.129:33388) 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf3fff640 1 -- 172.21.15.129:0/1677653996 learned_addr learned my addr 172.21.15.129:0/1677653996 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf88e3640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf4103440 0x7fcbf41043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf3fff640 1 -- 172.21.15.129:0/1677653996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbf4102a80 msgr2=0x7fcbf4103e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf3fff640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbf4102a80 0x7fcbf4103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf3fff640 1 -- 172.21.15.129:0/1677653996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf4103440 msgr2=0x7fcbf41043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf3fff640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf4103440 0x7fcbf41043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf3fff640 1 -- 172.21.15.129:0/1677653996 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf4104c30 con 0x7fcbf4101890 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf88e3640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf4103440 0x7fcbf41043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.791+0000 7fcbf3fff640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4101890 0x7fcbf4105820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400e9c0 tx=0x7fcbe400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.792+0000 7fcbf17fa640 1 -- 172.21.15.129:0/1677653996 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400cd80 con 0x7fcbf4101890 2024-04-26T21:16:07.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.792+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf41a2d00 con 0x7fcbf4101890 2024-04-26T21:16:07.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.792+0000 7fcbf17fa640 1 -- 172.21.15.129:0/1677653996 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbe40035f0 con 0x7fcbf4101890 2024-04-26T21:16:07.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.792+0000 7fcbf17fa640 1 -- 172.21.15.129:0/1677653996 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4010690 con 0x7fcbf4101890 2024-04-26T21:16:07.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.792+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf41a31a0 con 0x7fcbf4101890 2024-04-26T21:16:07.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.793+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf41a3430 con 0x7fcbf4101890 2024-04-26T21:16:07.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.794+0000 7fcbf17fa640 1 -- 172.21.15.129:0/1677653996 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fcbe40037a0 con 0x7fcbf4101890 2024-04-26T21:16:07.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.794+0000 7fcbf17fa640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcbdc0a3bb0 0x7fcbdc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.794+0000 7fcbf37fe640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcbdc0a3bb0 0x7fcbdc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.794+0000 7fcbf17fa640 1 -- 172.21.15.129:0/1677653996 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fcbe40d08a0 con 0x7fcbf4101890 2024-04-26T21:16:07.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.795+0000 7fcbf37fe640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcbdc0a3bb0 0x7fcbdc0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8005fd0 tx=0x7fcbe8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:07.798+0000 7fcbf17fa640 1 -- 172.21.15.129:0/1677653996 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcbe409d3d0 con 0x7fcbf4101890 2024-04-26T21:16:08.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.014+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fcbf41a37c0 con 0x7fcbf4101890 2024-04-26T21:16:08.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:08 smithi150 ceph-mon[32290]: pgmap v292: 97 pgs: 14 creating+peering, 24 unknown, 59 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-04-26T21:16:08.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:08 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3841885426' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:16:08.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:08 smithi150 ceph-mon[32290]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:16:08.535 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:08 smithi150 ceph-mon[32290]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T21:16:08.535 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:08 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1677653996' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:16:08.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.674+0000 7fcbf17fa640 1 -- 172.21.15.129:0/1677653996 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7fcbe409ccd0 con 0x7fcbf4101890 2024-04-26T21:16:08.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.676+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcbdc0a3bb0 msgr2=0x7fcbdc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:08.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.676+0000 7fcbfa346640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcbdc0a3bb0 0x7fcbdc0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8005fd0 tx=0x7fcbe8005850 comp rx=0 tx=0).stop 2024-04-26T21:16:08.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.676+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4101890 msgr2=0x7fcbf4105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:08.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.676+0000 7fcbfa346640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4101890 0x7fcbf4105820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400e9c0 tx=0x7fcbe400ee80 comp rx=0 tx=0).stop 2024-04-26T21:16:08.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.677+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 shutdown_connections 2024-04-26T21:16:08.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.677+0000 7fcbfa346640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcbdc0a3bb0 0x7fcbdc0a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.677+0000 7fcbfa346640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbf4103440 0x7fcbf41043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.677+0000 7fcbfa346640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbf4102a80 0x7fcbf4103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.677+0000 7fcbfa346640 1 --2- 172.21.15.129:0/1677653996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcbf4101890 0x7fcbf4105820 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.677+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 >> 172.21.15.129:0/1677653996 conn(0x7fcbf40fce20 msgr2=0x7fcbf40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:08.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.677+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 shutdown_connections 2024-04-26T21:16:08.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.677+0000 7fcbfa346640 1 -- 172.21.15.129:0/1677653996 wait complete. 2024-04-26T21:16:08.690 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:08 smithi129 ceph-mon[29237]: pgmap v292: 97 pgs: 14 creating+peering, 24 unknown, 59 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-04-26T21:16:08.690 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:08 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3841885426' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:16:08.690 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:08 smithi129 ceph-mon[29237]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:16:08.690 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:08 smithi129 ceph-mon[29237]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T21:16:08.690 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:08 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1677653996' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:16:08.691 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-26T21:16:08.691 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-26T21:16:08.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:08 smithi171 ceph-mon[33478]: pgmap v292: 97 pgs: 14 creating+peering, 24 unknown, 59 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-04-26T21:16:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:08 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3841885426' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:16:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:08 smithi171 ceph-mon[33478]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:16:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:08 smithi171 ceph-mon[33478]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T21:16:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:08 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1677653996' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:16:08.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.795+0000 7f68b4122640 1 -- 172.21.15.129:0/2325668697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac103400 msgr2=0x7f68ac1057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:08.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.795+0000 7f68b4122640 1 --2- 172.21.15.129:0/2325668697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac103400 0x7f68ac1057f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009900 tx=0x7f68a802a410 comp rx=0 tx=0).stop 2024-04-26T21:16:08.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.795+0000 7f68b4122640 1 -- 172.21.15.129:0/2325668697 shutdown_connections 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.795+0000 7f68b4122640 1 --2- 172.21.15.129:0/2325668697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac105d30 0x7f68ac108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.795+0000 7f68b4122640 1 --2- 172.21.15.129:0/2325668697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68ac108660 0x7f68ac10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.795+0000 7f68b4122640 1 --2- 172.21.15.129:0/2325668697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac103400 0x7f68ac1057f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.795+0000 7f68b4122640 1 -- 172.21.15.129:0/2325668697 >> 172.21.15.129:0/2325668697 conn(0x7f68ac0fce30 msgr2=0x7f68ac0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.795+0000 7f68b4122640 1 -- 172.21.15.129:0/2325668697 shutdown_connections 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.795+0000 7f68b4122640 1 -- 172.21.15.129:0/2325668697 wait complete. 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b4122640 1 Processor -- start 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b4122640 1 -- start start 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b4122640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68ac103400 0x7f68ac198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b4122640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac105d30 0x7f68ac1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b4122640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac108660 0x7f68ac1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b4122640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac10e4b0 con 0x7f68ac108660 2024-04-26T21:16:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b4122640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac10e330 con 0x7f68ac105d30 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b4122640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac10e630 con 0x7f68ac103400 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b26bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac108660 0x7f68ac1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b26bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac108660 0x7f68ac1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33412/0 (socket says 172.21.15.129:33412) 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b26bf640 1 -- 172.21.15.129:0/1678276250 learned_addr learned my addr 172.21.15.129:0/1678276250 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b16bd640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac105d30 0x7f68ac1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b1ebe640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68ac103400 0x7f68ac198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b26bf640 1 -- 172.21.15.129:0/1678276250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68ac103400 msgr2=0x7f68ac198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b26bf640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68ac103400 0x7f68ac198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b26bf640 1 -- 172.21.15.129:0/1678276250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac105d30 msgr2=0x7f68ac1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b26bf640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac105d30 0x7f68ac1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b26bf640 1 -- 172.21.15.129:0/1678276250 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac1a0830 con 0x7f68ac108660 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b26bf640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac108660 0x7f68ac1a0190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009fd0 tx=0x7f68a400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68a2ffd640 1 -- 172.21.15.129:0/1678276250 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4017070 con 0x7f68ac108660 2024-04-26T21:16:08.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68a2ffd640 1 -- 172.21.15.129:0/1678276250 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a4003e20 con 0x7f68ac108660 2024-04-26T21:16:08.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ac1a0b20 con 0x7f68ac108660 2024-04-26T21:16:08.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.796+0000 7f68a2ffd640 1 -- 172.21.15.129:0/1678276250 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4010460 con 0x7f68ac108660 2024-04-26T21:16:08.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.797+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ac1a1070 con 0x7f68ac108660 2024-04-26T21:16:08.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.798+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ac0fe920 con 0x7f68ac108660 2024-04-26T21:16:08.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.798+0000 7f68a2ffd640 1 -- 172.21.15.129:0/1678276250 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f68a4003970 con 0x7f68ac108660 2024-04-26T21:16:08.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.799+0000 7f68a2ffd640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68980a3c30 0x7f68980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:08.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.799+0000 7f68b1ebe640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68980a3c30 0x7f68980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:08.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.799+0000 7f68a2ffd640 1 -- 172.21.15.129:0/1678276250 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f68a40cffd0 con 0x7f68ac108660 2024-04-26T21:16:08.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.799+0000 7f68b1ebe640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68980a3c30 0x7f68980a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009230 tx=0x7f68a80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:08.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:08.804+0000 7f68a2ffd640 1 -- 172.21.15.129:0/1678276250 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f68a409c4b0 con 0x7f68ac108660 2024-04-26T21:16:09.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.009+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f68ac062d70 con 0x7f68980a3c30 2024-04-26T21:16:09.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.026+0000 7f68a2ffd640 1 -- 172.21.15.129:0/1678276250 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f68ac062d70 con 0x7f68980a3c30 2024-04-26T21:16:09.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68980a3c30 msgr2=0x7f68980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68980a3c30 0x7f68980a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009230 tx=0x7f68a80314a0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac108660 msgr2=0x7f68ac1a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac108660 0x7f68ac1a0190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009fd0 tx=0x7f68a400b260 comp rx=0 tx=0).stop 2024-04-26T21:16:09.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 shutdown_connections 2024-04-26T21:16:09.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68980a3c30 0x7f68980a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68ac105d30 0x7f68ac1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68ac108660 0x7f68ac1a0190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 --2- 172.21.15.129:0/1678276250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68ac103400 0x7f68ac198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 >> 172.21.15.129:0/1678276250 conn(0x7f68ac0fce30 msgr2=0x7f68ac103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:09.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 shutdown_connections 2024-04-26T21:16:09.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.029+0000 7f68b4122640 1 -- 172.21.15.129:0/1678276250 wait complete. 2024-04-26T21:16:09.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T21:16:09.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 -- 172.21.15.129:0/1055339137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 msgr2=0x7f476c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 --2- 172.21.15.129:0/1055339137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 0x7f476c109cd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4764009ca0 tx=0x7f476402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 -- 172.21.15.129:0/1055339137 shutdown_connections 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 --2- 172.21.15.129:0/1055339137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 0x7f476c109cd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 --2- 172.21.15.129:0/1055339137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f476c102a80 0x7f476c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 --2- 172.21.15.129:0/1055339137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f476c101890 0x7f476c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 -- 172.21.15.129:0/1055339137 >> 172.21.15.129:0/1055339137 conn(0x7f476c0fce00 msgr2=0x7f476c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 -- 172.21.15.129:0/1055339137 shutdown_connections 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 -- 172.21.15.129:0/1055339137 wait complete. 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.142+0000 7f4772bb4640 1 Processor -- start 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4772bb4640 1 -- start start 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4772bb4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f476c101890 0x7f476c06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4772bb4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f476c102a80 0x7f476c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4772bb4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 0x7f476c19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4772bb4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f476c10d3c0 con 0x7f476c103440 2024-04-26T21:16:09.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4772bb4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f476c10d240 con 0x7f476c102a80 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4772bb4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f476c10d540 con 0x7f476c101890 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4771151640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 0x7f476c19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4771151640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 0x7f476c19fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33464/0 (socket says 172.21.15.129:33464) 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4771151640 1 -- 172.21.15.129:0/3492793455 learned_addr learned my addr 172.21.15.129:0/3492793455 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f476bfff640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f476c102a80 0x7f476c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4770950640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f476c101890 0x7f476c06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4771151640 1 -- 172.21.15.129:0/3492793455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f476c101890 msgr2=0x7f476c06baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4771151640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f476c101890 0x7f476c06baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4771151640 1 -- 172.21.15.129:0/3492793455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f476c102a80 msgr2=0x7f476c19d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4771151640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f476c102a80 0x7f476c19d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4771151640 1 -- 172.21.15.129:0/3492793455 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c1a07f0 con 0x7f476c103440 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4770950640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f476c101890 0x7f476c06baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4771151640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 0x7f476c19fee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f47640097b0 tx=0x7f4764031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:09.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4769ffb640 1 -- 172.21.15.129:0/3492793455 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764003bf0 con 0x7f476c103440 2024-04-26T21:16:09.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c1a0a20 con 0x7f476c103440 2024-04-26T21:16:09.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4769ffb640 1 -- 172.21.15.129:0/3492793455 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4764003d90 con 0x7f476c103440 2024-04-26T21:16:09.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.143+0000 7f4769ffb640 1 -- 172.21.15.129:0/3492793455 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476402fc70 con 0x7f476c103440 2024-04-26T21:16:09.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.144+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1a0f70 con 0x7f476c103440 2024-04-26T21:16:09.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.145+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476c108bb0 con 0x7f476c103440 2024-04-26T21:16:09.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.146+0000 7f4769ffb640 1 -- 172.21.15.129:0/3492793455 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4764004040 con 0x7f476c103440 2024-04-26T21:16:09.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.146+0000 7f4769ffb640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f475c0a3bb0 0x7f475c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.146+0000 7f4770950640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f475c0a3bb0 0x7f475c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.146+0000 7f4769ffb640 1 -- 172.21.15.129:0/3492793455 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f47640f18f0 con 0x7f476c103440 2024-04-26T21:16:09.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.147+0000 7f4770950640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f475c0a3bb0 0x7f475c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f476c1028e0 tx=0x7f4760005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:09.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.151+0000 7f4769ffb640 1 -- 172.21.15.129:0/3492793455 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f47640be420 con 0x7f476c103440 2024-04-26T21:16:09.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.354+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f476c062e00 con 0x7f475c0a3bb0 2024-04-26T21:16:09.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.378+0000 7f4769ffb640 1 -- 172.21.15.129:0/3492793455 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f476c062e00 con 0x7f475c0a3bb0 2024-04-26T21:16:09.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f475c0a3bb0 msgr2=0x7f475c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f475c0a3bb0 0x7f475c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f476c1028e0 tx=0x7f4760005f90 comp rx=0 tx=0).stop 2024-04-26T21:16:09.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 msgr2=0x7f476c19fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 0x7f476c19fee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f47640097b0 tx=0x7f4764031840 comp rx=0 tx=0).stop 2024-04-26T21:16:09.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 shutdown_connections 2024-04-26T21:16:09.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f475c0a3bb0 0x7f475c0a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f476c102a80 0x7f476c19d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f476c103440 0x7f476c19fee0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 --2- 172.21.15.129:0/3492793455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f476c101890 0x7f476c06baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 >> 172.21.15.129:0/3492793455 conn(0x7f476c0fce00 msgr2=0x7f476c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:09.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 shutdown_connections 2024-04-26T21:16:09.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.380+0000 7f4772bb4640 1 -- 172.21.15.129:0/3492793455 wait complete. 2024-04-26T21:16:09.391 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:16:09.013+0000 7f27743b1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:09.391 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:16:09.013+0000 7f27743b1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:09.391 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:16:09.013+0000 7f27743b1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:09.391 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:16:09.013+0000 7f27743b1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:09.391 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:16:09.013+0000 7f27743b1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:09.392 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T21:16:09.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 -- 172.21.15.129:0/1053660889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414102a80 msgr2=0x7fc414102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 --2- 172.21.15.129:0/1053660889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414102a80 0x7fc414102f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc408009900 tx=0x7fc40802a410 comp rx=0 tx=0).stop 2024-04-26T21:16:09.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 -- 172.21.15.129:0/1053660889 shutdown_connections 2024-04-26T21:16:09.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 --2- 172.21.15.129:0/1053660889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc414103440 0x7fc414109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 --2- 172.21.15.129:0/1053660889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414102a80 0x7fc414102f00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 --2- 172.21.15.129:0/1053660889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414101890 0x7fc414101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 -- 172.21.15.129:0/1053660889 >> 172.21.15.129:0/1053660889 conn(0x7fc4140fce00 msgr2=0x7fc4140ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 -- 172.21.15.129:0/1053660889 shutdown_connections 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 -- 172.21.15.129:0/1053660889 wait complete. 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.494+0000 7fc419e1d640 1 Processor -- start 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc419e1d640 1 -- start start 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc419e1d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414101890 0x7fc414198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc419e1d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc414102a80 0x7fc414199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc419e1d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414103440 0x7fc4141a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc419e1d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410e4e0 con 0x7fc414103440 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc419e1d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410e360 con 0x7fc414101890 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc419e1d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410e660 con 0x7fc414102a80 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc413fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414103440 0x7fc4141a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc413fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414103440 0x7fc4141a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33476/0 (socket says 172.21.15.129:33476) 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc413fff640 1 -- 172.21.15.129:0/2112089279 learned_addr learned my addr 172.21.15.129:0/2112089279 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:09.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc4137fe640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414101890 0x7fc414198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc412ffd640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc414102a80 0x7fc414199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc413fff640 1 -- 172.21.15.129:0/2112089279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc414102a80 msgr2=0x7fc414199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc413fff640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc414102a80 0x7fc414199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc413fff640 1 -- 172.21.15.129:0/2112089279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414101890 msgr2=0x7fc414198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc413fff640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414101890 0x7fc414198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc413fff640 1 -- 172.21.15.129:0/2112089279 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4141a07e0 con 0x7fc414103440 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc413fff640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414103440 0x7fc4141a0170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00eaa0 tx=0x7fc40c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc410ff9640 1 -- 172.21.15.129:0/2112089279 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c00cdb0 con 0x7fc414103440 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.495+0000 7fc410ff9640 1 -- 172.21.15.129:0/2112089279 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40c005020 con 0x7fc414103440 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.496+0000 7fc410ff9640 1 -- 172.21.15.129:0/2112089279 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0116d0 con 0x7fc414103440 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.496+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4141a0ad0 con 0x7fc414103440 2024-04-26T21:16:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.496+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4141a1070 con 0x7fc414103440 2024-04-26T21:16:09.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.496+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc414108c80 con 0x7fc414103440 2024-04-26T21:16:09.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.497+0000 7fc410ff9640 1 -- 172.21.15.129:0/2112089279 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc40c0051d0 con 0x7fc414103440 2024-04-26T21:16:09.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.498+0000 7fc410ff9640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc3fc0a3c30 0x7fc3fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.498+0000 7fc4137fe640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc3fc0a3c30 0x7fc3fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.498+0000 7fc410ff9640 1 -- 172.21.15.129:0/2112089279 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc40c0d0400 con 0x7fc414103440 2024-04-26T21:16:09.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.499+0000 7fc4137fe640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc3fc0a3c30 0x7fc3fc0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc4141028e0 tx=0x7fc404005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:09.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.503+0000 7fc410ff9640 1 -- 172.21.15.129:0/2112089279 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc40c016030 con 0x7fc414103440 2024-04-26T21:16:09.632 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:16:09 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b[62955]: 2024-04-26T21:16:09.349+0000 7fd7aeb1d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:16:09.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.694+0000 7fc410ff9640 1 -- 172.21.15.129:0/2112089279 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc40c098ea0 con 0x7fc414103440 2024-04-26T21:16:09.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.717+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fc414062d70 con 0x7fc3fc0a3c30 2024-04-26T21:16:09.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.722+0000 7fc410ff9640 1 -- 172.21.15.129:0/2112089279 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc414062d70 con 0x7fc3fc0a3c30 2024-04-26T21:16:09.723 INFO:teuthology.orchestra.run.smithi129.stdout:/volumes/qa/sv_0/4241ac65-aef1-46f9-b6d9-6be5128e193d 2024-04-26T21:16:09.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc3fc0a3c30 msgr2=0x7fc3fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc3fc0a3c30 0x7fc3fc0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc4141028e0 tx=0x7fc404005f90 comp rx=0 tx=0).stop 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414103440 msgr2=0x7fc4141a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414103440 0x7fc4141a0170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00eaa0 tx=0x7fc40c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 shutdown_connections 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc3fc0a3c30 0x7fc3fc0a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc414103440 0x7fc4141a0170 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc414102a80 0x7fc414199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 --2- 172.21.15.129:0/2112089279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414101890 0x7fc414198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 >> 172.21.15.129:0/2112089279 conn(0x7fc4140fce00 msgr2=0x7fc414105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 shutdown_connections 2024-04-26T21:16:09.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.724+0000 7fc419e1d640 1 -- 172.21.15.129:0/2112089279 wait complete. 2024-04-26T21:16:09.736 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T21:16:09.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.837+0000 7f43d7872640 1 -- 172.21.15.129:0/2563448536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0102a80 msgr2=0x7f43d0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.837+0000 7f43d7872640 1 --2- 172.21.15.129:0/2563448536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0102a80 0x7f43d0102f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009900 tx=0x7f43c002a410 comp rx=0 tx=0).stop 2024-04-26T21:16:09.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.837+0000 7f43d7872640 1 -- 172.21.15.129:0/2563448536 shutdown_connections 2024-04-26T21:16:09.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.837+0000 7f43d7872640 1 --2- 172.21.15.129:0/2563448536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d0103440 0x7f43d0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.837+0000 7f43d7872640 1 --2- 172.21.15.129:0/2563448536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0102a80 0x7f43d0102f00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.837+0000 7f43d7872640 1 --2- 172.21.15.129:0/2563448536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d0101890 0x7f43d0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.837+0000 7f43d7872640 1 -- 172.21.15.129:0/2563448536 >> 172.21.15.129:0/2563448536 conn(0x7f43d00fce00 msgr2=0x7f43d00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:09.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 -- 172.21.15.129:0/2563448536 shutdown_connections 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 -- 172.21.15.129:0/2563448536 wait complete. 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 Processor -- start 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 -- start start 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0101890 0x7f43d006bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d0102a80 0x7f43d019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d0103440 0x7f43d019ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43d010d3c0 con 0x7f43d0101890 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43d010d240 con 0x7f43d0102a80 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d7872640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f43d010d540 con 0x7f43d0103440 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d560e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0101890 0x7f43d006bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d560e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0101890 0x7f43d006bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33504/0 (socket says 172.21.15.129:33504) 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d560e640 1 -- 172.21.15.129:0/947113956 learned_addr learned my addr 172.21.15.129:0/947113956 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.838+0000 7f43d4e0d640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d0102a80 0x7f43d019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43d5e0f640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d0103440 0x7f43d019ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43d560e640 1 -- 172.21.15.129:0/947113956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d0103440 msgr2=0x7f43d019ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43d560e640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d0103440 0x7f43d019ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43d560e640 1 -- 172.21.15.129:0/947113956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d0102a80 msgr2=0x7f43d019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43d560e640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d0102a80 0x7f43d019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43d560e640 1 -- 172.21.15.129:0/947113956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d01a07f0 con 0x7f43d0101890 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43d560e640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0101890 0x7f43d006bac0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00d770 tx=0x7f43cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43c67fc640 1 -- 172.21.15.129:0/947113956 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc017070 con 0x7f43d0101890 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43c67fc640 1 -- 172.21.15.129:0/947113956 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc009c30 con 0x7f43d0101890 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43c67fc640 1 -- 172.21.15.129:0/947113956 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc003cc0 con 0x7f43d0101890 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d01a0ae0 con 0x7f43d0101890 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.839+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d01a1000 con 0x7f43d0101890 2024-04-26T21:16:09.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.840+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d0108c50 con 0x7f43d0101890 2024-04-26T21:16:09.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.841+0000 7f43c67fc640 1 -- 172.21.15.129:0/947113956 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f43cc003e60 con 0x7f43d0101890 2024-04-26T21:16:09.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.841+0000 7f43c67fc640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f43bc0a3c40 0x7f43bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:09.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.842+0000 7f43d4e0d640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f43bc0a3c40 0x7f43bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:09.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.842+0000 7f43c67fc640 1 -- 172.21.15.129:0/947113956 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f43cc0cfd90 con 0x7f43d0101890 2024-04-26T21:16:09.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.842+0000 7f43d4e0d640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f43bc0a3c40 0x7f43bc0a6100 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000c00 tx=0x7f43c00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:09.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:09.846+0000 7f43c67fc640 1 -- 172.21.15.129:0/947113956 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f43cc09c7c0 con 0x7f43d0101890 2024-04-26T21:16:09.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1677653996' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:16:09.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-mon[29237]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T21:16:09.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-mon[29237]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:09.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-mon[29237]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-26T21:16:09.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:09 smithi129 ceph-mon[29237]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:09.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:09 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1677653996' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:16:09.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:09 smithi171 ceph-mon[33478]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T21:16:09.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:09 smithi171 ceph-mon[33478]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:09.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:09 smithi171 ceph-mon[33478]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-26T21:16:09.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:09 smithi171 ceph-mon[33478]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:10.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.051+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f43d0062d70 con 0x7f43bc0a3c40 2024-04-26T21:16:10.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.078+0000 7f43c67fc640 1 -- 172.21.15.129:0/947113956 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f43d0062d70 con 0x7f43bc0a3c40 2024-04-26T21:16:10.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f43bc0a3c40 msgr2=0x7f43bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f43bc0a3c40 0x7f43bc0a6100 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000c00 tx=0x7f43c00314a0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0101890 msgr2=0x7f43d006bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0101890 0x7f43d006bac0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00d770 tx=0x7f43cc00dc30 comp rx=0 tx=0).stop 2024-04-26T21:16:10.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 shutdown_connections 2024-04-26T21:16:10.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f43bc0a3c40 0x7f43bc0a6100 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d0102a80 0x7f43d019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d0103440 0x7f43d019ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 --2- 172.21.15.129:0/947113956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d0101890 0x7f43d006bac0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 >> 172.21.15.129:0/947113956 conn(0x7f43d00fce00 msgr2=0x7f43d0105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:10.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 shutdown_connections 2024-04-26T21:16:10.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.080+0000 7f43d7872640 1 -- 172.21.15.129:0/947113956 wait complete. 2024-04-26T21:16:10.092 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T21:16:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:09 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1677653996' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:16:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:09 smithi150 ceph-mon[32290]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T21:16:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:09 smithi150 ceph-mon[32290]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:09 smithi150 ceph-mon[32290]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-26T21:16:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:09 smithi150 ceph-mon[32290]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 -- 172.21.15.129:0/3970550927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0102a80 msgr2=0x7fd4e0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/3970550927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0102a80 0x7fd4e0102f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009900 tx=0x7fd4d402a410 comp rx=0 tx=0).stop 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 -- 172.21.15.129:0/3970550927 shutdown_connections 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/3970550927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0103440 0x7fd4e0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/3970550927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0102a80 0x7fd4e0102f00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/3970550927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4e0101890 0x7fd4e0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 -- 172.21.15.129:0/3970550927 >> 172.21.15.129:0/3970550927 conn(0x7fd4e00fce00 msgr2=0x7fd4e00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 -- 172.21.15.129:0/3970550927 shutdown_connections 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 -- 172.21.15.129:0/3970550927 wait complete. 2024-04-26T21:16:10.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 Processor -- start 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.195+0000 7fd4e61a4640 1 -- start start 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4e61a4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0101890 0x7fd4e006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4e61a4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0102a80 0x7fd4e019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4e61a4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4e0103440 0x7fd4e019ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4e61a4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e010d3a0 con 0x7fd4e0101890 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4e61a4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e010d220 con 0x7fd4e0102a80 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4e61a4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e010d520 con 0x7fd4e0103440 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4df7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0101890 0x7fd4e006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4df7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0101890 0x7fd4e006baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33524/0 (socket says 172.21.15.129:33524) 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4df7fe640 1 -- 172.21.15.129:0/1120294916 learned_addr learned my addr 172.21.15.129:0/1120294916 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4dffff640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4e0103440 0x7fd4e019ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4deffd640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0102a80 0x7fd4e019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4df7fe640 1 -- 172.21.15.129:0/1120294916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4e0103440 msgr2=0x7fd4e019ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4df7fe640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4e0103440 0x7fd4e019ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4df7fe640 1 -- 172.21.15.129:0/1120294916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0102a80 msgr2=0x7fd4e019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4df7fe640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0102a80 0x7fd4e019d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4df7fe640 1 -- 172.21.15.129:0/1120294916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e01a0860 con 0x7fd4e0101890 2024-04-26T21:16:10.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4deffd640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0102a80 0x7fd4e019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:10.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4df7fe640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0101890 0x7fd4e006baa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000d950 tx=0x7fd4d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:10.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4dcff9640 1 -- 172.21.15.129:0/1120294916 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0017020 con 0x7fd4e0101890 2024-04-26T21:16:10.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4dcff9640 1 -- 172.21.15.129:0/1120294916 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4d0009c30 con 0x7fd4e0101890 2024-04-26T21:16:10.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e01a0af0 con 0x7fd4e0101890 2024-04-26T21:16:10.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4dcff9640 1 -- 172.21.15.129:0/1120294916 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000f6b0 con 0x7fd4e0101890 2024-04-26T21:16:10.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.196+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e01a1040 con 0x7fd4e0101890 2024-04-26T21:16:10.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.197+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e0108cd0 con 0x7fd4e0101890 2024-04-26T21:16:10.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.199+0000 7fd4dcff9640 1 -- 172.21.15.129:0/1120294916 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd4d001a020 con 0x7fd4e0101890 2024-04-26T21:16:10.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.199+0000 7fd4dcff9640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd4c80a3c40 0x7fd4c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.199+0000 7fd4deffd640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd4c80a3c40 0x7fd4c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.199+0000 7fd4dcff9640 1 -- 172.21.15.129:0/1120294916 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd4d00d0050 con 0x7fd4e0101890 2024-04-26T21:16:10.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.199+0000 7fd4deffd640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd4c80a3c40 0x7fd4c80a6100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd4e006b810 tx=0x7fd4d40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:10.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.204+0000 7fd4dcff9640 1 -- 172.21.15.129:0/1120294916 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd4d0009de0 con 0x7fd4e0101890 2024-04-26T21:16:10.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.409+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd4e0062e00 con 0x7fd4c80a3c40 2024-04-26T21:16:10.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.414+0000 7fd4dcff9640 1 -- 172.21.15.129:0/1120294916 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd4e0062e00 con 0x7fd4c80a3c40 2024-04-26T21:16:10.415 INFO:teuthology.orchestra.run.smithi129.stdout:/volumes/qa/sv_1/9733ba9c-39b7-458b-9c70-cf2aa87c2e0a 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd4c80a3c40 msgr2=0x7fd4c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd4c80a3c40 0x7fd4c80a6100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd4e006b810 tx=0x7fd4d40314a0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0101890 msgr2=0x7fd4e006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0101890 0x7fd4e006baa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000d950 tx=0x7fd4d000de10 comp rx=0 tx=0).stop 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 shutdown_connections 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd4c80a3c40 0x7fd4c80a6100 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0102a80 0x7fd4e019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4e0103440 0x7fd4e019ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 --2- 172.21.15.129:0/1120294916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4e0101890 0x7fd4e006baa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.416+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 >> 172.21.15.129:0/1120294916 conn(0x7fd4e00fce00 msgr2=0x7fd4e0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:10.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.417+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 shutdown_connections 2024-04-26T21:16:10.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.417+0000 7fd4e61a4640 1 -- 172.21.15.129:0/1120294916 wait complete. 2024-04-26T21:16:10.429 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:16:10.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.525+0000 7fda94738640 1 -- 172.21.15.129:0/18122511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c103b70 msgr2=0x7fda8c103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.525+0000 7fda94738640 1 --2- 172.21.15.129:0/18122511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c103b70 0x7fda8c103ff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009900 tx=0x7fda7c02a410 comp rx=0 tx=0).stop 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.525+0000 7fda94738640 1 -- 172.21.15.129:0/18122511 shutdown_connections 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.525+0000 7fda94738640 1 --2- 172.21.15.129:0/18122511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda8c104530 0x7fda8c10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.525+0000 7fda94738640 1 --2- 172.21.15.129:0/18122511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c103b70 0x7fda8c103ff0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.525+0000 7fda94738640 1 --2- 172.21.15.129:0/18122511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda8c102980 0x7fda8c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.525+0000 7fda94738640 1 -- 172.21.15.129:0/18122511 >> 172.21.15.129:0/18122511 conn(0x7fda8c0fdf10 msgr2=0x7fda8c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.525+0000 7fda94738640 1 -- 172.21.15.129:0/18122511 shutdown_connections 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.525+0000 7fda94738640 1 -- 172.21.15.129:0/18122511 wait complete. 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda94738640 1 Processor -- start 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda94738640 1 -- start start 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda94738640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda8c102980 0x7fda8c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda94738640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda8c103b70 0x7fda8c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda94738640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c104530 0x7fda8c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda94738640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10e4b0 con 0x7fda8c104530 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda94738640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10e330 con 0x7fda8c103b70 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda94738640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10e630 con 0x7fda8c102980 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda92cd5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c104530 0x7fda8c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda92cd5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c104530 0x7fda8c1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33540/0 (socket says 172.21.15.129:33540) 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda92cd5640 1 -- 172.21.15.129:0/2592194901 learned_addr learned my addr 172.21.15.129:0/2592194901 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda924d4640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda8c102980 0x7fda8c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda91cd3640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda8c103b70 0x7fda8c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda92cd5640 1 -- 172.21.15.129:0/2592194901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda8c102980 msgr2=0x7fda8c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda92cd5640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda8c102980 0x7fda8c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda92cd5640 1 -- 172.21.15.129:0/2592194901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda8c103b70 msgr2=0x7fda8c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda92cd5640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda8c103b70 0x7fda8c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda92cd5640 1 -- 172.21.15.129:0/2592194901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c1a0830 con 0x7fda8c104530 2024-04-26T21:16:10.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.526+0000 7fda92cd5640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c104530 0x7fda8c1a0190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fda84009fd0 tx=0x7fda8400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.527+0000 7fda837fe640 1 -- 172.21.15.129:0/2592194901 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84017070 con 0x7fda8c104530 2024-04-26T21:16:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.527+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c1a0b20 con 0x7fda8c104530 2024-04-26T21:16:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.527+0000 7fda837fe640 1 -- 172.21.15.129:0/2592194901 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda84003e20 con 0x7fda8c104530 2024-04-26T21:16:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.527+0000 7fda837fe640 1 -- 172.21.15.129:0/2592194901 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84010460 con 0x7fda8c104530 2024-04-26T21:16:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.527+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c1a1020 con 0x7fda8c104530 2024-04-26T21:16:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.527+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c109d70 con 0x7fda8c104530 2024-04-26T21:16:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.528+0000 7fda837fe640 1 -- 172.21.15.129:0/2592194901 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fda84003970 con 0x7fda8c104530 2024-04-26T21:16:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.529+0000 7fda837fe640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fda780a3c40 0x7fda780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.529+0000 7fda924d4640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fda780a3c40 0x7fda780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.529+0000 7fda837fe640 1 -- 172.21.15.129:0/2592194901 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fda840d01f0 con 0x7fda8c104530 2024-04-26T21:16:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.530+0000 7fda924d4640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fda780a3c40 0x7fda780a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fda8c19a0f0 tx=0x7fda8800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:10.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.534+0000 7fda837fe640 1 -- 172.21.15.129:0/2592194901 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fda8409c550 con 0x7fda8c104530 2024-04-26T21:16:10.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.749+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda8c02dad0 con 0x7fda8c104530 2024-04-26T21:16:10.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.750+0000 7fda837fe640 1 -- 172.21.15.129:0/2592194901 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7fda840a0850 con 0x7fda8c104530 2024-04-26T21:16:10.751 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:10.751 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:16:10.752 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fda780a3c40 msgr2=0x7fda780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fda780a3c40 0x7fda780a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fda8c19a0f0 tx=0x7fda8800b390 comp rx=0 tx=0).stop 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c104530 msgr2=0x7fda8c1a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c104530 0x7fda8c1a0190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fda84009fd0 tx=0x7fda8400b260 comp rx=0 tx=0).stop 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 shutdown_connections 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fda780a3c40 0x7fda780a6100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda8c103b70 0x7fda8c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda8c104530 0x7fda8c1a0190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 --2- 172.21.15.129:0/2592194901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda8c102980 0x7fda8c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 >> 172.21.15.129:0/2592194901 conn(0x7fda8c0fdf10 msgr2=0x7fda8c106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:10.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.752+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 shutdown_connections 2024-04-26T21:16:10.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.753+0000 7fda94738640 1 -- 172.21.15.129:0/2592194901 wait complete. 2024-04-26T21:16:10.765 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:16:10.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.861+0000 7ff3b5904640 1 -- 172.21.15.129:0/3307202073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0103430 msgr2=0x7ff3b0109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.861+0000 7ff3b5904640 1 --2- 172.21.15.129:0/3307202073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0103430 0x7ff3b0109cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009900 tx=0x7ff3a002a410 comp rx=0 tx=0).stop 2024-04-26T21:16:10.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 -- 172.21.15.129:0/3307202073 shutdown_connections 2024-04-26T21:16:10.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 --2- 172.21.15.129:0/3307202073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0102a70 0x7ff3b0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 --2- 172.21.15.129:0/3307202073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0103430 0x7ff3b0109cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 --2- 172.21.15.129:0/3307202073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3b0101880 0x7ff3b0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 -- 172.21.15.129:0/3307202073 >> 172.21.15.129:0/3307202073 conn(0x7ff3b00fce10 msgr2=0x7ff3b00ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 -- 172.21.15.129:0/3307202073 shutdown_connections 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 -- 172.21.15.129:0/3307202073 wait complete. 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 Processor -- start 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 -- start start 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0101880 0x7ff3b006bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3b0102a70 0x7ff3b019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.862+0000 7ff3b5904640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0103430 0x7ff3b019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3b5904640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010d3b0 con 0x7ff3b0101880 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3b5904640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010d230 con 0x7ff3b0103430 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3b5904640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010d530 con 0x7ff3b0102a70 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3aeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0101880 0x7ff3b006bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3aeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0101880 0x7ff3b006bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33556/0 (socket says 172.21.15.129:33556) 2024-04-26T21:16:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3aeffd640 1 -- 172.21.15.129:0/1811534847 learned_addr learned my addr 172.21.15.129:0/1811534847 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3ae7fc640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3b0102a70 0x7ff3b019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3af7fe640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0103430 0x7ff3b019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3aeffd640 1 -- 172.21.15.129:0/1811534847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3b0102a70 msgr2=0x7ff3b019d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3aeffd640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3b0102a70 0x7ff3b019d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3aeffd640 1 -- 172.21.15.129:0/1811534847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0103430 msgr2=0x7ff3b019fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3aeffd640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0103430 0x7ff3b019fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3aeffd640 1 -- 172.21.15.129:0/1811534847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b01a0780 con 0x7ff3b0101880 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3aeffd640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0101880 0x7ff3b006bac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800e9c0 tx=0x7ff3a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3b4902640 1 -- 172.21.15.129:0/1811534847 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800cd80 con 0x7ff3b0101880 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b01a0a70 con 0x7ff3b0101880 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3b4902640 1 -- 172.21.15.129:0/1811534847 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a80035f0 con 0x7ff3b0101880 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3b4902640 1 -- 172.21.15.129:0/1811534847 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8010690 con 0x7ff3b0101880 2024-04-26T21:16:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.863+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b01a0f70 con 0x7ff3b0101880 2024-04-26T21:16:10.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.864+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b0108bc0 con 0x7ff3b0101880 2024-04-26T21:16:10.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.865+0000 7ff3b4902640 1 -- 172.21.15.129:0/1811534847 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff3a8004000 con 0x7ff3b0101880 2024-04-26T21:16:10.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.866+0000 7ff3b4902640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff3980a3c40 0x7ff3980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:10.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.866+0000 7ff3ae7fc640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff3980a3c40 0x7ff3980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:10.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.866+0000 7ff3b4902640 1 -- 172.21.15.129:0/1811534847 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff3a80d0980 con 0x7ff3b0101880 2024-04-26T21:16:10.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.866+0000 7ff3ae7fc640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff3980a3c40 0x7ff3980a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff3b006b830 tx=0x7ff3a4009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:10.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:10.870+0000 7ff3b4902640 1 -- 172.21.15.129:0/1811534847 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff3a809d3b0 con 0x7ff3b0101880 2024-04-26T21:16:10.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:10 smithi129 ceph-mon[29237]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T21:16:10.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:10 smithi129 ceph-mon[29237]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:10.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:10 smithi129 ceph-mon[29237]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:10.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:10 smithi171 ceph-mon[33478]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T21:16:10.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:10 smithi171 ceph-mon[33478]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:10.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:10 smithi171 ceph-mon[33478]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:11.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.054+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3b002dd40 con 0x7ff3b0101880 2024-04-26T21:16:11.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.055+0000 7ff3b4902640 1 -- 172.21.15.129:0/1811534847 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22274 (secure 0 0 0) 0x7ff3a809ccb0 con 0x7ff3b0101880 2024-04-26T21:16:11.057 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:11.057 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:04.683012+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff3980a3c40 msgr2=0x7ff3980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff3980a3c40 0x7ff3980a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff3b006b830 tx=0x7ff3a4009680 comp rx=0 tx=0).stop 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0101880 msgr2=0x7ff3b006bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0101880 0x7ff3b006bac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800e9c0 tx=0x7ff3a800ee80 comp rx=0 tx=0).stop 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 shutdown_connections 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff3980a3c40 0x7ff3980a6100 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0103430 0x7ff3b019fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3b0102a70 0x7ff3b019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 --2- 172.21.15.129:0/1811534847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3b0101880 0x7ff3b006bac0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.057+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 >> 172.21.15.129:0/1811534847 conn(0x7ff3b00fce10 msgr2=0x7ff3b0105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:11.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.058+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 shutdown_connections 2024-04-26T21:16:11.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.058+0000 7ff3b5904640 1 -- 172.21.15.129:0/1811534847 wait complete. 2024-04-26T21:16:11.070 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:16:11.071 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:16:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:10 smithi150 ceph-mon[32290]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T21:16:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:10 smithi150 ceph-mon[32290]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:10 smithi150 ceph-mon[32290]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:11.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 -- 172.21.15.129:0/4171419172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0103430 msgr2=0x7f44f0109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/4171419172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0103430 0x7f44f0109cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009ca0 tx=0x7f44e802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 -- 172.21.15.129:0/4171419172 shutdown_connections 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/4171419172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0103430 0x7f44f0109cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/4171419172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44f0102a70 0x7f44f0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/4171419172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0101880 0x7f44f0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 -- 172.21.15.129:0/4171419172 >> 172.21.15.129:0/4171419172 conn(0x7f44f00fce30 msgr2=0x7f44f00ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 -- 172.21.15.129:0/4171419172 shutdown_connections 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 -- 172.21.15.129:0/4171419172 wait complete. 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 Processor -- start 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 -- start start 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44f0101880 0x7f44f0198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0102a70 0x7f44f01990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:11.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0103430 0x7f44f01a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44f010d3b0 con 0x7f44f0102a70 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44f010d230 con 0x7f44f0103430 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44f6b5f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f44f010d530 con 0x7f44f0101880 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.168+0000 7f44effff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0102a70 0x7f44f01990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44effff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0102a70 0x7f44f01990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33570/0 (socket says 172.21.15.129:33570) 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44effff640 1 -- 172.21.15.129:0/2195720182 learned_addr learned my addr 172.21.15.129:0/2195720182 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44f48fb640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44f0101880 0x7f44f0198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44f50fc640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0103430 0x7f44f01a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44effff640 1 -- 172.21.15.129:0/2195720182 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44f0101880 msgr2=0x7f44f0198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44effff640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44f0101880 0x7f44f0198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44effff640 1 -- 172.21.15.129:0/2195720182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0103430 msgr2=0x7f44f01a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44effff640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0103430 0x7f44f01a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44effff640 1 -- 172.21.15.129:0/2195720182 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f01a0790 con 0x7f44f0102a70 2024-04-26T21:16:11.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44f50fc640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0103430 0x7f44f01a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:11.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44effff640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0102a70 0x7f44f01990b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e9f0 tx=0x7f44dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:11.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44edffb640 1 -- 172.21.15.129:0/2195720182 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc00cd80 con 0x7f44f0102a70 2024-04-26T21:16:11.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f01a0a80 con 0x7f44f0102a70 2024-04-26T21:16:11.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44edffb640 1 -- 172.21.15.129:0/2195720182 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44dc0054d0 con 0x7f44f0102a70 2024-04-26T21:16:11.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44edffb640 1 -- 172.21.15.129:0/2195720182 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc010690 con 0x7f44f0102a70 2024-04-26T21:16:11.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.169+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f01a0fd0 con 0x7f44f0102a70 2024-04-26T21:16:11.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.170+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f01a1260 con 0x7f44f0102a70 2024-04-26T21:16:11.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.171+0000 7f44edffb640 1 -- 172.21.15.129:0/2195720182 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f44dc004e90 con 0x7f44f0102a70 2024-04-26T21:16:11.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.172+0000 7f44edffb640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f44e00a3c40 0x7f44e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:11.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.172+0000 7f44f48fb640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f44e00a3c40 0x7f44e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:11.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.172+0000 7f44edffb640 1 -- 172.21.15.129:0/2195720182 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f44dc0d0870 con 0x7f44f0102a70 2024-04-26T21:16:11.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.172+0000 7f44f48fb640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f44e00a3c40 0x7f44e00a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f44f01028d0 tx=0x7f44e400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:11.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.176+0000 7f44edffb640 1 -- 172.21.15.129:0/2195720182 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f44dc09d2a0 con 0x7f44f0102a70 2024-04-26T21:16:11.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.355+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44f006b6e0 con 0x7f44e00a3c40 2024-04-26T21:16:11.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.362+0000 7f44edffb640 1 -- 172.21.15.129:0/2195720182 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258833 (secure 0 0 0) 0x7f44f006b6e0 con 0x7f44e00a3c40 2024-04-26T21:16:11.368 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:11.370 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f44e00a3c40 msgr2=0x7f44e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f44e00a3c40 0x7f44e00a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f44f01028d0 tx=0x7f44e400b390 comp rx=0 tx=0).stop 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0102a70 msgr2=0x7f44f01990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0102a70 0x7f44f01990b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e9f0 tx=0x7f44dc00edf0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 shutdown_connections 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f44e00a3c40 0x7f44e00a6100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0103430 0x7f44f01a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44f0102a70 0x7f44f01990b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 --2- 172.21.15.129:0/2195720182 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44f0101880 0x7f44f0198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 >> 172.21.15.129:0/2195720182 conn(0x7f44f00fce30 msgr2=0x7f44f006afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 shutdown_connections 2024-04-26T21:16:11.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.365+0000 7f44f6b5f640 1 -- 172.21.15.129:0/2195720182 wait complete. 2024-04-26T21:16:11.377 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-26T21:16:11.276575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":134,"num_read_kb":116,"num_write":226,"num_write_kb":2952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":250,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360312,"kb_used_data":11076,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581512,"statfs":{"total":1151940427776,"available":1151571468288,"internally_reserved":0,"allocated":11341824,"data_stored":7113896,"data_compressed":108620,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001622"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.179037+0000","last_change":"2024-04-26T21:16:03.700581+0000","last_active":"2024-04-26T21:16:05.179037+0000","last_peered":"2024-04-26T21:16:05.179037+0000","last_clean":"2024-04-26T21:16:05.179037+0000","last_became_active":"2024-04-26T21:16:03.700280+0000","last_became_peered":"2024-04-26T21:16:03.700280+0000","last_unstale":"2024-04-26T21:16:05.179037+0000","last_undegraded":"2024-04-26T21:16:05.179037+0000","last_fullsized":"2024-04-26T21:16:05.179037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:01.329784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991539+0000","last_change":"2024-04-26T21:16:03.702058+0000","last_active":"2024-04-26T21:16:04.991539+0000","last_peered":"2024-04-26T21:16:04.991539+0000","last_clean":"2024-04-26T21:16:04.991539+0000","last_became_active":"2024-04-26T21:16:03.701728+0000","last_became_peered":"2024-04-26T21:16:03.701728+0000","last_unstale":"2024-04-26T21:16:04.991539+0000","last_undegraded":"2024-04-26T21:16:04.991539+0000","last_fullsized":"2024-04-26T21:16:04.991539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:12.718225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991160+0000","last_change":"2024-04-26T21:16:03.710461+0000","last_active":"2024-04-26T21:16:04.991160+0000","last_peered":"2024-04-26T21:16:04.991160+0000","last_clean":"2024-04-26T21:16:04.991160+0000","last_became_active":"2024-04-26T21:16:03.710130+0000","last_became_peered":"2024-04-26T21:16:03.710130+0000","last_unstale":"2024-04-26T21:16:04.991160+0000","last_undegraded":"2024-04-26T21:16:04.991160+0000","last_fullsized":"2024-04-26T21:16:04.991160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:21.294677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.905131+0000","last_change":"2024-04-26T21:16:03.706836+0000","last_active":"2024-04-26T21:16:04.905131+0000","last_peered":"2024-04-26T21:16:04.905131+0000","last_clean":"2024-04-26T21:16:04.905131+0000","last_became_active":"2024-04-26T21:16:03.706515+0000","last_became_peered":"2024-04-26T21:16:03.706515+0000","last_unstale":"2024-04-26T21:16:04.905131+0000","last_undegraded":"2024-04-26T21:16:04.905131+0000","last_fullsized":"2024-04-26T21:16:04.905131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:56.198337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990275+0000","last_change":"2024-04-26T21:16:03.704865+0000","last_active":"2024-04-26T21:16:04.990275+0000","last_peered":"2024-04-26T21:16:04.990275+0000","last_clean":"2024-04-26T21:16:04.990275+0000","last_became_active":"2024-04-26T21:16:03.704560+0000","last_became_peered":"2024-04-26T21:16:03.704560+0000","last_unstale":"2024-04-26T21:16:04.990275+0000","last_undegraded":"2024-04-26T21:16:04.990275+0000","last_fullsized":"2024-04-26T21:16:04.990275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:51:03.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692636+0000","last_change":"2024-04-26T21:16:03.700265+0000","last_active":"2024-04-26T21:16:04.692636+0000","last_peered":"2024-04-26T21:16:04.692636+0000","last_clean":"2024-04-26T21:16:04.692636+0000","last_became_active":"2024-04-26T21:16:03.699878+0000","last_became_peered":"2024-04-26T21:16:03.699878+0000","last_unstale":"2024-04-26T21:16:04.692636+0000","last_undegraded":"2024-04-26T21:16:04.692636+0000","last_fullsized":"2024-04-26T21:16:04.692636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:12.907254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690185+0000","last_change":"2024-04-26T21:16:03.697569+0000","last_active":"2024-04-26T21:16:04.690185+0000","last_peered":"2024-04-26T21:16:04.690185+0000","last_clean":"2024-04-26T21:16:04.690185+0000","last_became_active":"2024-04-26T21:16:03.697262+0000","last_became_peered":"2024-04-26T21:16:03.697262+0000","last_unstale":"2024-04-26T21:16:04.690185+0000","last_undegraded":"2024-04-26T21:16:04.690185+0000","last_fullsized":"2024-04-26T21:16:04.690185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:28.645750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990718+0000","last_change":"2024-04-26T21:16:03.705788+0000","last_active":"2024-04-26T21:16:04.990718+0000","last_peered":"2024-04-26T21:16:04.990718+0000","last_clean":"2024-04-26T21:16:04.990718+0000","last_became_active":"2024-04-26T21:16:03.705479+0000","last_became_peered":"2024-04-26T21:16:03.705479+0000","last_unstale":"2024-04-26T21:16:04.990718+0000","last_undegraded":"2024-04-26T21:16:04.990718+0000","last_fullsized":"2024-04-26T21:16:04.990718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:27.306777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.695162+0000","last_change":"2024-04-26T21:16:03.709716+0000","last_active":"2024-04-26T21:16:04.695162+0000","last_peered":"2024-04-26T21:16:04.695162+0000","last_clean":"2024-04-26T21:16:04.695162+0000","last_became_active":"2024-04-26T21:16:03.709219+0000","last_became_peered":"2024-04-26T21:16:03.709219+0000","last_unstale":"2024-04-26T21:16:04.695162+0000","last_undegraded":"2024-04-26T21:16:04.695162+0000","last_fullsized":"2024-04-26T21:16:04.695162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:18.936075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.693066+0000","last_change":"2024-04-26T21:16:03.694745+0000","last_active":"2024-04-26T21:16:04.693066+0000","last_peered":"2024-04-26T21:16:04.693066+0000","last_clean":"2024-04-26T21:16:04.693066+0000","last_became_active":"2024-04-26T21:16:03.694044+0000","last_became_peered":"2024-04-26T21:16:03.694044+0000","last_unstale":"2024-04-26T21:16:04.693066+0000","last_undegraded":"2024-04-26T21:16:04.693066+0000","last_fullsized":"2024-04-26T21:16:04.693066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:13.060467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690607+0000","last_change":"2024-04-26T21:16:03.706434+0000","last_active":"2024-04-26T21:16:04.690607+0000","last_peered":"2024-04-26T21:16:04.690607+0000","last_clean":"2024-04-26T21:16:04.690607+0000","last_became_active":"2024-04-26T21:16:03.706093+0000","last_became_peered":"2024-04-26T21:16:03.706093+0000","last_unstale":"2024-04-26T21:16:04.690607+0000","last_undegraded":"2024-04-26T21:16:04.690607+0000","last_fullsized":"2024-04-26T21:16:04.690607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:04.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.905135+0000","last_change":"2024-04-26T21:16:03.707672+0000","last_active":"2024-04-26T21:16:04.905135+0000","last_peered":"2024-04-26T21:16:04.905135+0000","last_clean":"2024-04-26T21:16:04.905135+0000","last_became_active":"2024-04-26T21:16:03.707344+0000","last_became_peered":"2024-04-26T21:16:03.707344+0000","last_unstale":"2024-04-26T21:16:04.905135+0000","last_undegraded":"2024-04-26T21:16:04.905135+0000","last_fullsized":"2024-04-26T21:16:04.905135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:07.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687931+0000","last_change":"2024-04-26T21:16:03.699816+0000","last_active":"2024-04-26T21:16:04.687931+0000","last_peered":"2024-04-26T21:16:04.687931+0000","last_clean":"2024-04-26T21:16:04.687931+0000","last_became_active":"2024-04-26T21:16:03.699514+0000","last_became_peered":"2024-04-26T21:16:03.699514+0000","last_unstale":"2024-04-26T21:16:04.687931+0000","last_undegraded":"2024-04-26T21:16:04.687931+0000","last_fullsized":"2024-04-26T21:16:04.687931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:59.230438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990746+0000","last_change":"2024-04-26T21:16:03.709627+0000","last_active":"2024-04-26T21:16:04.990746+0000","last_peered":"2024-04-26T21:16:04.990746+0000","last_clean":"2024-04-26T21:16:04.990746+0000","last_became_active":"2024-04-26T21:16:03.709238+0000","last_became_peered":"2024-04-26T21:16:03.709238+0000","last_unstale":"2024-04-26T21:16:04.990746+0000","last_undegraded":"2024-04-26T21:16:04.990746+0000","last_fullsized":"2024-04-26T21:16:04.990746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:55.303059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990402+0000","last_change":"2024-04-26T21:16:03.702275+0000","last_active":"2024-04-26T21:16:04.990402+0000","last_peered":"2024-04-26T21:16:04.990402+0000","last_clean":"2024-04-26T21:16:04.990402+0000","last_became_active":"2024-04-26T21:16:03.701966+0000","last_became_peered":"2024-04-26T21:16:03.701966+0000","last_unstale":"2024-04-26T21:16:04.990402+0000","last_undegraded":"2024-04-26T21:16:04.990402+0000","last_fullsized":"2024-04-26T21:16:04.990402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:02.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694686+0000","last_change":"2024-04-26T21:16:03.711979+0000","last_active":"2024-04-26T21:16:04.694686+0000","last_peered":"2024-04-26T21:16:04.694686+0000","last_clean":"2024-04-26T21:16:04.694686+0000","last_became_active":"2024-04-26T21:16:03.711637+0000","last_became_peered":"2024-04-26T21:16:03.711637+0000","last_unstale":"2024-04-26T21:16:04.694686+0000","last_undegraded":"2024-04-26T21:16:04.694686+0000","last_fullsized":"2024-04-26T21:16:04.694686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:27.817993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.179039+0000","last_change":"2024-04-26T21:16:03.701303+0000","last_active":"2024-04-26T21:16:05.179039+0000","last_peered":"2024-04-26T21:16:05.179039+0000","last_clean":"2024-04-26T21:16:05.179039+0000","last_became_active":"2024-04-26T21:16:03.701001+0000","last_became_peered":"2024-04-26T21:16:03.701001+0000","last_unstale":"2024-04-26T21:16:05.179039+0000","last_undegraded":"2024-04-26T21:16:05.179039+0000","last_fullsized":"2024-04-26T21:16:05.179039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:08.349915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990812+0000","last_change":"2024-04-26T21:16:03.696775+0000","last_active":"2024-04-26T21:16:04.990812+0000","last_peered":"2024-04-26T21:16:04.990812+0000","last_clean":"2024-04-26T21:16:04.990812+0000","last_became_active":"2024-04-26T21:16:03.696433+0000","last_became_peered":"2024-04-26T21:16:03.696433+0000","last_unstale":"2024-04-26T21:16:04.990812+0000","last_undegraded":"2024-04-26T21:16:04.990812+0000","last_fullsized":"2024-04-26T21:16:04.990812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:50.029169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691414+0000","last_change":"2024-04-26T21:16:03.698188+0000","last_active":"2024-04-26T21:16:04.691414+0000","last_peered":"2024-04-26T21:16:04.691414+0000","last_clean":"2024-04-26T21:16:04.691414+0000","last_became_active":"2024-04-26T21:16:03.697882+0000","last_became_peered":"2024-04-26T21:16:03.697882+0000","last_unstale":"2024-04-26T21:16:04.691414+0000","last_undegraded":"2024-04-26T21:16:04.691414+0000","last_fullsized":"2024-04-26T21:16:04.691414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:04.737619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990899+0000","last_change":"2024-04-26T21:16:03.275296+0000","last_active":"2024-04-26T21:16:04.990899+0000","last_peered":"2024-04-26T21:16:04.990899+0000","last_clean":"2024-04-26T21:16:04.990899+0000","last_became_active":"2024-04-26T21:16:03.274733+0000","last_became_peered":"2024-04-26T21:16:03.274733+0000","last_unstale":"2024-04-26T21:16:04.990899+0000","last_undegraded":"2024-04-26T21:16:04.990899+0000","last_fullsized":"2024-04-26T21:16:04.990899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:16.512934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991644+0000","last_change":"2024-04-26T21:16:03.705823+0000","last_active":"2024-04-26T21:16:04.991644+0000","last_peered":"2024-04-26T21:16:04.991644+0000","last_clean":"2024-04-26T21:16:04.991644+0000","last_became_active":"2024-04-26T21:16:03.705512+0000","last_became_peered":"2024-04-26T21:16:03.705512+0000","last_unstale":"2024-04-26T21:16:04.991644+0000","last_undegraded":"2024-04-26T21:16:04.991644+0000","last_fullsized":"2024-04-26T21:16:04.991644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:17.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991832+0000","last_change":"2024-04-26T21:16:03.285008+0000","last_active":"2024-04-26T21:16:04.991832+0000","last_peered":"2024-04-26T21:16:04.991832+0000","last_clean":"2024-04-26T21:16:04.991832+0000","last_became_active":"2024-04-26T21:16:03.284679+0000","last_became_peered":"2024-04-26T21:16:03.284679+0000","last_unstale":"2024-04-26T21:16:04.991832+0000","last_undegraded":"2024-04-26T21:16:04.991832+0000","last_fullsized":"2024-04-26T21:16:04.991832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:35.404086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.992144+0000","last_change":"2024-04-26T21:16:03.710571+0000","last_active":"2024-04-26T21:16:04.992144+0000","last_peered":"2024-04-26T21:16:04.992144+0000","last_clean":"2024-04-26T21:16:04.992144+0000","last_became_active":"2024-04-26T21:16:03.710237+0000","last_became_peered":"2024-04-26T21:16:03.710237+0000","last_unstale":"2024-04-26T21:16:04.992144+0000","last_undegraded":"2024-04-26T21:16:04.992144+0000","last_fullsized":"2024-04-26T21:16:04.992144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:53.188554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990535+0000","last_change":"2024-04-26T21:16:03.280177+0000","last_active":"2024-04-26T21:16:04.990535+0000","last_peered":"2024-04-26T21:16:04.990535+0000","last_clean":"2024-04-26T21:16:04.990535+0000","last_became_active":"2024-04-26T21:16:03.279881+0000","last_became_peered":"2024-04-26T21:16:03.279881+0000","last_unstale":"2024-04-26T21:16:04.990535+0000","last_undegraded":"2024-04-26T21:16:04.990535+0000","last_fullsized":"2024-04-26T21:16:04.990535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:41.489019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694801+0000","last_change":"2024-04-26T21:16:03.711355+0000","last_active":"2024-04-26T21:16:04.694801+0000","last_peered":"2024-04-26T21:16:04.694801+0000","last_clean":"2024-04-26T21:16:04.694801+0000","last_became_active":"2024-04-26T21:16:03.710885+0000","last_became_peered":"2024-04-26T21:16:03.710885+0000","last_unstale":"2024-04-26T21:16:04.694801+0000","last_undegraded":"2024-04-26T21:16:04.694801+0000","last_fullsized":"2024-04-26T21:16:04.694801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:10.515590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990429+0000","last_change":"2024-04-26T21:16:03.278484+0000","last_active":"2024-04-26T21:16:04.990429+0000","last_peered":"2024-04-26T21:16:04.990429+0000","last_clean":"2024-04-26T21:16:04.990429+0000","last_became_active":"2024-04-26T21:16:03.277402+0000","last_became_peered":"2024-04-26T21:16:03.277402+0000","last_unstale":"2024-04-26T21:16:04.990429+0000","last_undegraded":"2024-04-26T21:16:04.990429+0000","last_fullsized":"2024-04-26T21:16:04.990429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:21.807942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689994+0000","last_change":"2024-04-26T21:16:03.713577+0000","last_active":"2024-04-26T21:16:04.689994+0000","last_peered":"2024-04-26T21:16:04.689994+0000","last_clean":"2024-04-26T21:16:04.689994+0000","last_became_active":"2024-04-26T21:16:03.713318+0000","last_became_peered":"2024-04-26T21:16:03.713318+0000","last_unstale":"2024-04-26T21:16:04.689994+0000","last_undegraded":"2024-04-26T21:16:04.689994+0000","last_fullsized":"2024-04-26T21:16:04.689994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:57.888552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689718+0000","last_change":"2024-04-26T21:16:03.277219+0000","last_active":"2024-04-26T21:16:04.689718+0000","last_peered":"2024-04-26T21:16:04.689718+0000","last_clean":"2024-04-26T21:16:04.689718+0000","last_became_active":"2024-04-26T21:16:03.276885+0000","last_became_peered":"2024-04-26T21:16:03.276885+0000","last_unstale":"2024-04-26T21:16:04.689718+0000","last_undegraded":"2024-04-26T21:16:04.689718+0000","last_fullsized":"2024-04-26T21:16:04.689718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:45.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990316+0000","last_change":"2024-04-26T21:16:03.703453+0000","last_active":"2024-04-26T21:16:04.990316+0000","last_peered":"2024-04-26T21:16:04.990316+0000","last_clean":"2024-04-26T21:16:04.990316+0000","last_became_active":"2024-04-26T21:16:03.703161+0000","last_became_peered":"2024-04-26T21:16:03.703161+0000","last_unstale":"2024-04-26T21:16:04.990316+0000","last_undegraded":"2024-04-26T21:16:04.990316+0000","last_fullsized":"2024-04-26T21:16:04.990316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.999243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692475+0000","last_change":"2024-04-26T21:16:03.274815+0000","last_active":"2024-04-26T21:16:04.692475+0000","last_peered":"2024-04-26T21:16:04.692475+0000","last_clean":"2024-04-26T21:16:04.692475+0000","last_became_active":"2024-04-26T21:16:03.274509+0000","last_became_peered":"2024-04-26T21:16:03.274509+0000","last_unstale":"2024-04-26T21:16:04.692475+0000","last_undegraded":"2024-04-26T21:16:04.692475+0000","last_fullsized":"2024-04-26T21:16:04.692475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:27.212714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689700+0000","last_change":"2024-04-26T21:16:03.700986+0000","last_active":"2024-04-26T21:16:04.689700+0000","last_peered":"2024-04-26T21:16:04.689700+0000","last_clean":"2024-04-26T21:16:04.689700+0000","last_became_active":"2024-04-26T21:16:03.700630+0000","last_became_peered":"2024-04-26T21:16:03.700630+0000","last_unstale":"2024-04-26T21:16:04.689700+0000","last_undegraded":"2024-04-26T21:16:04.689700+0000","last_fullsized":"2024-04-26T21:16:04.689700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:12.703732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687693+0000","last_change":"2024-04-26T21:16:03.270417+0000","last_active":"2024-04-26T21:16:04.687693+0000","last_peered":"2024-04-26T21:16:04.687693+0000","last_clean":"2024-04-26T21:16:04.687693+0000","last_became_active":"2024-04-26T21:16:03.270066+0000","last_became_peered":"2024-04-26T21:16:03.270066+0000","last_unstale":"2024-04-26T21:16:04.687693+0000","last_undegraded":"2024-04-26T21:16:04.687693+0000","last_fullsized":"2024-04-26T21:16:04.687693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:46.794698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990564+0000","last_change":"2024-04-26T21:16:03.700221+0000","last_active":"2024-04-26T21:16:04.990564+0000","last_peered":"2024-04-26T21:16:04.990564+0000","last_clean":"2024-04-26T21:16:04.990564+0000","last_became_active":"2024-04-26T21:16:03.699798+0000","last_became_peered":"2024-04-26T21:16:03.699798+0000","last_unstale":"2024-04-26T21:16:04.990564+0000","last_undegraded":"2024-04-26T21:16:04.990564+0000","last_fullsized":"2024-04-26T21:16:04.990564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:52.083700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689324+0000","last_change":"2024-04-26T21:16:03.282355+0000","last_active":"2024-04-26T21:16:04.689324+0000","last_peered":"2024-04-26T21:16:04.689324+0000","last_clean":"2024-04-26T21:16:04.689324+0000","last_became_active":"2024-04-26T21:16:03.281169+0000","last_became_peered":"2024-04-26T21:16:03.281169+0000","last_unstale":"2024-04-26T21:16:04.689324+0000","last_undegraded":"2024-04-26T21:16:04.689324+0000","last_fullsized":"2024-04-26T21:16:04.689324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:55.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991575+0000","last_change":"2024-04-26T21:16:03.701016+0000","last_active":"2024-04-26T21:16:04.991575+0000","last_peered":"2024-04-26T21:16:04.991575+0000","last_clean":"2024-04-26T21:16:04.991575+0000","last_became_active":"2024-04-26T21:16:03.700711+0000","last_became_peered":"2024-04-26T21:16:03.700711+0000","last_unstale":"2024-04-26T21:16:04.991575+0000","last_undegraded":"2024-04-26T21:16:04.991575+0000","last_fullsized":"2024-04-26T21:16:04.991575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.390477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692871+0000","last_change":"2024-04-26T21:16:03.284948+0000","last_active":"2024-04-26T21:16:04.692871+0000","last_peered":"2024-04-26T21:16:04.692871+0000","last_clean":"2024-04-26T21:16:04.692871+0000","last_became_active":"2024-04-26T21:16:03.284570+0000","last_became_peered":"2024-04-26T21:16:03.284570+0000","last_unstale":"2024-04-26T21:16:04.692871+0000","last_undegraded":"2024-04-26T21:16:04.692871+0000","last_fullsized":"2024-04-26T21:16:04.692871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:22.883834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.695088+0000","last_change":"2024-04-26T21:16:03.710359+0000","last_active":"2024-04-26T21:16:04.695088+0000","last_peered":"2024-04-26T21:16:04.695088+0000","last_clean":"2024-04-26T21:16:04.695088+0000","last_became_active":"2024-04-26T21:16:03.710053+0000","last_became_peered":"2024-04-26T21:16:03.710053+0000","last_unstale":"2024-04-26T21:16:04.695088+0000","last_undegraded":"2024-04-26T21:16:04.695088+0000","last_fullsized":"2024-04-26T21:16:04.695088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:45.981083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690376+0000","last_change":"2024-04-26T21:16:03.278787+0000","last_active":"2024-04-26T21:16:04.690376+0000","last_peered":"2024-04-26T21:16:04.690376+0000","last_clean":"2024-04-26T21:16:04.690376+0000","last_became_active":"2024-04-26T21:16:03.278483+0000","last_became_peered":"2024-04-26T21:16:03.278483+0000","last_unstale":"2024-04-26T21:16:04.690376+0000","last_undegraded":"2024-04-26T21:16:04.690376+0000","last_fullsized":"2024-04-26T21:16:04.690376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:40.303372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.992068+0000","last_change":"2024-04-26T21:16:03.709501+0000","last_active":"2024-04-26T21:16:04.992068+0000","last_peered":"2024-04-26T21:16:04.992068+0000","last_clean":"2024-04-26T21:16:04.992068+0000","last_became_active":"2024-04-26T21:16:03.709118+0000","last_became_peered":"2024-04-26T21:16:03.709118+0000","last_unstale":"2024-04-26T21:16:04.992068+0000","last_undegraded":"2024-04-26T21:16:04.992068+0000","last_fullsized":"2024-04-26T21:16:04.992068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:23.904465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689984+0000","last_change":"2024-04-26T21:16:03.277387+0000","last_active":"2024-04-26T21:16:04.689984+0000","last_peered":"2024-04-26T21:16:04.689984+0000","last_clean":"2024-04-26T21:16:04.689984+0000","last_became_active":"2024-04-26T21:16:03.274765+0000","last_became_peered":"2024-04-26T21:16:03.274765+0000","last_unstale":"2024-04-26T21:16:04.689984+0000","last_undegraded":"2024-04-26T21:16:04.689984+0000","last_fullsized":"2024-04-26T21:16:04.689984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:51.081883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694930+0000","last_change":"2024-04-26T21:16:03.710977+0000","last_active":"2024-04-26T21:16:04.694930+0000","last_peered":"2024-04-26T21:16:04.694930+0000","last_clean":"2024-04-26T21:16:04.694930+0000","last_became_active":"2024-04-26T21:16:03.710284+0000","last_became_peered":"2024-04-26T21:16:03.710284+0000","last_unstale":"2024-04-26T21:16:04.694930+0000","last_undegraded":"2024-04-26T21:16:04.694930+0000","last_fullsized":"2024-04-26T21:16:04.694930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:46.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:07.680826+0000","last_change":"2024-04-26T21:16:03.276865+0000","last_active":"2024-04-26T21:16:07.680826+0000","last_peered":"2024-04-26T21:16:07.680826+0000","last_clean":"2024-04-26T21:16:07.680826+0000","last_became_active":"2024-04-26T21:16:03.276564+0000","last_became_peered":"2024-04-26T21:16:03.276564+0000","last_unstale":"2024-04-26T21:16:07.680826+0000","last_undegraded":"2024-04-26T21:16:07.680826+0000","last_fullsized":"2024-04-26T21:16:07.680826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:33.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990367+0000","last_change":"2024-04-26T21:16:03.700441+0000","last_active":"2024-04-26T21:16:04.990367+0000","last_peered":"2024-04-26T21:16:04.990367+0000","last_clean":"2024-04-26T21:16:04.990367+0000","last_became_active":"2024-04-26T21:16:03.700142+0000","last_became_peered":"2024-04-26T21:16:03.700142+0000","last_unstale":"2024-04-26T21:16:04.990367+0000","last_undegraded":"2024-04-26T21:16:04.990367+0000","last_fullsized":"2024-04-26T21:16:04.990367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:35.166603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:06.049054+0000","last_change":"2024-04-26T21:16:03.280144+0000","last_active":"2024-04-26T21:16:06.049054+0000","last_peered":"2024-04-26T21:16:06.049054+0000","last_clean":"2024-04-26T21:16:06.049054+0000","last_became_active":"2024-04-26T21:16:03.279844+0000","last_became_peered":"2024-04-26T21:16:03.279844+0000","last_unstale":"2024-04-26T21:16:06.049054+0000","last_undegraded":"2024-04-26T21:16:06.049054+0000","last_fullsized":"2024-04-26T21:16:06.049054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:00.704181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687987+0000","last_change":"2024-04-26T21:16:03.701732+0000","last_active":"2024-04-26T21:16:04.687987+0000","last_peered":"2024-04-26T21:16:04.687987+0000","last_clean":"2024-04-26T21:16:04.687987+0000","last_became_active":"2024-04-26T21:16:03.701424+0000","last_became_peered":"2024-04-26T21:16:03.701424+0000","last_unstale":"2024-04-26T21:16:04.687987+0000","last_undegraded":"2024-04-26T21:16:04.687987+0000","last_fullsized":"2024-04-26T21:16:04.687987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:02.293086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689250+0000","last_change":"2024-04-26T21:16:03.276816+0000","last_active":"2024-04-26T21:16:04.689250+0000","last_peered":"2024-04-26T21:16:04.689250+0000","last_clean":"2024-04-26T21:16:04.689250+0000","last_became_active":"2024-04-26T21:16:03.276471+0000","last_became_peered":"2024-04-26T21:16:03.276471+0000","last_unstale":"2024-04-26T21:16:04.689250+0000","last_undegraded":"2024-04-26T21:16:04.689250+0000","last_fullsized":"2024-04-26T21:16:04.689250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:44.630638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690511+0000","last_change":"2024-04-26T21:16:03.701002+0000","last_active":"2024-04-26T21:16:04.690511+0000","last_peered":"2024-04-26T21:16:04.690511+0000","last_clean":"2024-04-26T21:16:04.690511+0000","last_became_active":"2024-04-26T21:16:03.700649+0000","last_became_peered":"2024-04-26T21:16:03.700649+0000","last_unstale":"2024-04-26T21:16:04.690511+0000","last_undegraded":"2024-04-26T21:16:04.690511+0000","last_fullsized":"2024-04-26T21:16:04.690511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:45.571735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.904718+0000","last_change":"2024-04-26T21:16:03.273727+0000","last_active":"2024-04-26T21:16:04.904718+0000","last_peered":"2024-04-26T21:16:04.904718+0000","last_clean":"2024-04-26T21:16:04.904718+0000","last_became_active":"2024-04-26T21:16:03.273287+0000","last_became_peered":"2024-04-26T21:16:03.273287+0000","last_unstale":"2024-04-26T21:16:04.904718+0000","last_undegraded":"2024-04-26T21:16:04.904718+0000","last_fullsized":"2024-04-26T21:16:04.904718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:48.713646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690245+0000","last_change":"2024-04-26T21:16:03.696760+0000","last_active":"2024-04-26T21:16:04.690245+0000","last_peered":"2024-04-26T21:16:04.690245+0000","last_clean":"2024-04-26T21:16:04.690245+0000","last_became_active":"2024-04-26T21:16:03.696446+0000","last_became_peered":"2024-04-26T21:16:03.696446+0000","last_unstale":"2024-04-26T21:16:04.690245+0000","last_undegraded":"2024-04-26T21:16:04.690245+0000","last_fullsized":"2024-04-26T21:16:04.690245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:30.710360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990010+0000","last_change":"2024-04-26T21:16:03.282643+0000","last_active":"2024-04-26T21:16:04.990010+0000","last_peered":"2024-04-26T21:16:04.990010+0000","last_clean":"2024-04-26T21:16:04.990010+0000","last_became_active":"2024-04-26T21:16:03.282315+0000","last_became_peered":"2024-04-26T21:16:03.282315+0000","last_unstale":"2024-04-26T21:16:04.990010+0000","last_undegraded":"2024-04-26T21:16:04.990010+0000","last_fullsized":"2024-04-26T21:16:04.990010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:13.358002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689135+0000","last_change":"2024-04-26T21:16:03.711992+0000","last_active":"2024-04-26T21:16:04.689135+0000","last_peered":"2024-04-26T21:16:04.689135+0000","last_clean":"2024-04-26T21:16:04.689135+0000","last_became_active":"2024-04-26T21:16:03.711755+0000","last_became_peered":"2024-04-26T21:16:03.711755+0000","last_unstale":"2024-04-26T21:16:04.689135+0000","last_undegraded":"2024-04-26T21:16:04.689135+0000","last_fullsized":"2024-04-26T21:16:04.689135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:01.321530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990586+0000","last_change":"2024-04-26T21:16:03.274722+0000","last_active":"2024-04-26T21:16:04.990586+0000","last_peered":"2024-04-26T21:16:04.990586+0000","last_clean":"2024-04-26T21:16:04.990586+0000","last_became_active":"2024-04-26T21:16:03.274399+0000","last_became_peered":"2024-04-26T21:16:03.274399+0000","last_unstale":"2024-04-26T21:16:04.990586+0000","last_undegraded":"2024-04-26T21:16:04.990586+0000","last_fullsized":"2024-04-26T21:16:04.990586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:35.247456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178814+0000","last_change":"2024-04-26T21:16:03.698344+0000","last_active":"2024-04-26T21:16:05.178814+0000","last_peered":"2024-04-26T21:16:05.178814+0000","last_clean":"2024-04-26T21:16:05.178814+0000","last_became_active":"2024-04-26T21:16:03.698037+0000","last_became_peered":"2024-04-26T21:16:03.698037+0000","last_unstale":"2024-04-26T21:16:05.178814+0000","last_undegraded":"2024-04-26T21:16:05.178814+0000","last_fullsized":"2024-04-26T21:16:05.178814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.573421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990550+0000","last_change":"2024-04-26T21:16:03.276459+0000","last_active":"2024-04-26T21:16:04.990550+0000","last_peered":"2024-04-26T21:16:04.990550+0000","last_clean":"2024-04-26T21:16:04.990550+0000","last_became_active":"2024-04-26T21:16:03.276108+0000","last_became_peered":"2024-04-26T21:16:03.276108+0000","last_unstale":"2024-04-26T21:16:04.990550+0000","last_undegraded":"2024-04-26T21:16:04.990550+0000","last_fullsized":"2024-04-26T21:16:04.990550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.364925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692814+0000","last_change":"2024-04-26T21:16:03.696030+0000","last_active":"2024-04-26T21:16:04.692814+0000","last_peered":"2024-04-26T21:16:04.692814+0000","last_clean":"2024-04-26T21:16:04.692814+0000","last_became_active":"2024-04-26T21:16:03.695717+0000","last_became_peered":"2024-04-26T21:16:03.695717+0000","last_unstale":"2024-04-26T21:16:04.692814+0000","last_undegraded":"2024-04-26T21:16:04.692814+0000","last_fullsized":"2024-04-26T21:16:04.692814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:55.702216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690150+0000","last_change":"2024-04-26T21:16:03.269188+0000","last_active":"2024-04-26T21:16:04.690150+0000","last_peered":"2024-04-26T21:16:04.690150+0000","last_clean":"2024-04-26T21:16:04.690150+0000","last_became_active":"2024-04-26T21:16:03.268850+0000","last_became_peered":"2024-04-26T21:16:03.268850+0000","last_unstale":"2024-04-26T21:16:04.690150+0000","last_undegraded":"2024-04-26T21:16:04.690150+0000","last_fullsized":"2024-04-26T21:16:04.690150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:54.183612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692731+0000","last_change":"2024-04-26T21:16:03.700284+0000","last_active":"2024-04-26T21:16:04.692731+0000","last_peered":"2024-04-26T21:16:04.692731+0000","last_clean":"2024-04-26T21:16:04.692731+0000","last_became_active":"2024-04-26T21:16:03.699965+0000","last_became_peered":"2024-04-26T21:16:03.699965+0000","last_unstale":"2024-04-26T21:16:04.692731+0000","last_undegraded":"2024-04-26T21:16:04.692731+0000","last_fullsized":"2024-04-26T21:16:04.692731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:40.547296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689760+0000","last_change":"2024-04-26T21:16:03.701370+0000","last_active":"2024-04-26T21:16:04.689760+0000","last_peered":"2024-04-26T21:16:04.689760+0000","last_clean":"2024-04-26T21:16:04.689760+0000","last_became_active":"2024-04-26T21:16:03.700585+0000","last_became_peered":"2024-04-26T21:16:03.700585+0000","last_unstale":"2024-04-26T21:16:04.689760+0000","last_undegraded":"2024-04-26T21:16:04.689760+0000","last_fullsized":"2024-04-26T21:16:04.689760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.348009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687467+0000","last_change":"2024-04-26T21:16:03.281141+0000","last_active":"2024-04-26T21:16:04.687467+0000","last_peered":"2024-04-26T21:16:04.687467+0000","last_clean":"2024-04-26T21:16:04.687467+0000","last_became_active":"2024-04-26T21:16:03.280812+0000","last_became_peered":"2024-04-26T21:16:03.280812+0000","last_unstale":"2024-04-26T21:16:04.687467+0000","last_undegraded":"2024-04-26T21:16:04.687467+0000","last_fullsized":"2024-04-26T21:16:04.687467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:11.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991138+0000","last_change":"2024-04-26T21:16:03.711312+0000","last_active":"2024-04-26T21:16:04.991138+0000","last_peered":"2024-04-26T21:16:04.991138+0000","last_clean":"2024-04-26T21:16:04.991138+0000","last_became_active":"2024-04-26T21:16:03.711026+0000","last_became_peered":"2024-04-26T21:16:03.711026+0000","last_unstale":"2024-04-26T21:16:04.991138+0000","last_undegraded":"2024-04-26T21:16:04.991138+0000","last_fullsized":"2024-04-26T21:16:04.991138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:10.185877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.904868+0000","last_change":"2024-04-26T21:16:03.273428+0000","last_active":"2024-04-26T21:16:04.904868+0000","last_peered":"2024-04-26T21:16:04.904868+0000","last_clean":"2024-04-26T21:16:04.904868+0000","last_became_active":"2024-04-26T21:16:03.272854+0000","last_became_peered":"2024-04-26T21:16:03.272854+0000","last_unstale":"2024-04-26T21:16:04.904868+0000","last_undegraded":"2024-04-26T21:16:04.904868+0000","last_fullsized":"2024-04-26T21:16:04.904868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:48.735059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":336,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690028+0000","last_change":"2024-04-26T21:14:50.729892+0000","last_active":"2024-04-26T21:16:04.690028+0000","last_peered":"2024-04-26T21:16:04.690028+0000","last_clean":"2024-04-26T21:16:04.690028+0000","last_became_active":"2024-04-26T21:14:50.729430+0000","last_became_peered":"2024-04-26T21:14:50.729430+0000","last_unstale":"2024-04-26T21:16:04.690028+0000","last_undegraded":"2024-04-26T21:16:04.690028+0000","last_fullsized":"2024-04-26T21:16:04.690028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_clean_scrub_stamp":"2024-04-26T21:11:22.039289+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:13.510971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687729+0000","last_change":"2024-04-26T21:16:03.700702+0000","last_active":"2024-04-26T21:16:04.687729+0000","last_peered":"2024-04-26T21:16:04.687729+0000","last_clean":"2024-04-26T21:16:04.687729+0000","last_became_active":"2024-04-26T21:16:03.700412+0000","last_became_peered":"2024-04-26T21:16:03.700412+0000","last_unstale":"2024-04-26T21:16:04.687729+0000","last_undegraded":"2024-04-26T21:16:04.687729+0000","last_fullsized":"2024-04-26T21:16:04.687729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:49.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690256+0000","last_change":"2024-04-26T21:16:03.271730+0000","last_active":"2024-04-26T21:16:04.690256+0000","last_peered":"2024-04-26T21:16:04.690256+0000","last_clean":"2024-04-26T21:16:04.690256+0000","last_became_active":"2024-04-26T21:16:03.271401+0000","last_became_peered":"2024-04-26T21:16:03.271401+0000","last_unstale":"2024-04-26T21:16:04.690256+0000","last_undegraded":"2024-04-26T21:16:04.690256+0000","last_fullsized":"2024-04-26T21:16:04.690256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:16.469022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990304+0000","last_change":"2024-04-26T21:16:03.276229+0000","last_active":"2024-04-26T21:16:04.990304+0000","last_peered":"2024-04-26T21:16:04.990304+0000","last_clean":"2024-04-26T21:16:04.990304+0000","last_became_active":"2024-04-26T21:16:03.275910+0000","last_became_peered":"2024-04-26T21:16:03.275910+0000","last_unstale":"2024-04-26T21:16:04.990304+0000","last_undegraded":"2024-04-26T21:16:04.990304+0000","last_fullsized":"2024-04-26T21:16:04.990304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:03.932254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687898+0000","last_change":"2024-04-26T21:16:03.698293+0000","last_active":"2024-04-26T21:16:04.687898+0000","last_peered":"2024-04-26T21:16:04.687898+0000","last_clean":"2024-04-26T21:16:04.687898+0000","last_became_active":"2024-04-26T21:16:03.697935+0000","last_became_peered":"2024-04-26T21:16:03.697935+0000","last_unstale":"2024-04-26T21:16:04.687898+0000","last_undegraded":"2024-04-26T21:16:04.687898+0000","last_fullsized":"2024-04-26T21:16:04.687898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:54.292921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.682425+0000","last_change":"2024-04-26T21:16:03.275162+0000","last_active":"2024-04-26T21:16:05.682425+0000","last_peered":"2024-04-26T21:16:05.682425+0000","last_clean":"2024-04-26T21:16:05.682425+0000","last_became_active":"2024-04-26T21:16:03.274791+0000","last_became_peered":"2024-04-26T21:16:03.274791+0000","last_unstale":"2024-04-26T21:16:05.682425+0000","last_undegraded":"2024-04-26T21:16:05.682425+0000","last_fullsized":"2024-04-26T21:16:05.682425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:15.583747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.992098+0000","last_change":"2024-04-26T21:16:03.701473+0000","last_active":"2024-04-26T21:16:04.992098+0000","last_peered":"2024-04-26T21:16:04.992098+0000","last_clean":"2024-04-26T21:16:04.992098+0000","last_became_active":"2024-04-26T21:16:03.701070+0000","last_became_peered":"2024-04-26T21:16:03.701070+0000","last_unstale":"2024-04-26T21:16:04.992098+0000","last_undegraded":"2024-04-26T21:16:04.992098+0000","last_fullsized":"2024-04-26T21:16:04.992098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:25.373301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.905090+0000","last_change":"2024-04-26T21:16:03.271036+0000","last_active":"2024-04-26T21:16:04.905090+0000","last_peered":"2024-04-26T21:16:04.905090+0000","last_clean":"2024-04-26T21:16:04.905090+0000","last_became_active":"2024-04-26T21:16:03.270686+0000","last_became_peered":"2024-04-26T21:16:03.270686+0000","last_unstale":"2024-04-26T21:16:04.905090+0000","last_undegraded":"2024-04-26T21:16:04.905090+0000","last_fullsized":"2024-04-26T21:16:04.905090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:21.025757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991688+0000","last_change":"2024-04-26T21:16:03.705314+0000","last_active":"2024-04-26T21:16:04.991688+0000","last_peered":"2024-04-26T21:16:04.991688+0000","last_clean":"2024-04-26T21:16:04.991688+0000","last_became_active":"2024-04-26T21:16:03.704952+0000","last_became_peered":"2024-04-26T21:16:03.704952+0000","last_unstale":"2024-04-26T21:16:04.991688+0000","last_undegraded":"2024-04-26T21:16:04.991688+0000","last_fullsized":"2024-04-26T21:16:04.991688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:57.363909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694488+0000","last_change":"2024-04-26T21:16:03.280995+0000","last_active":"2024-04-26T21:16:04.694488+0000","last_peered":"2024-04-26T21:16:04.694488+0000","last_clean":"2024-04-26T21:16:04.694488+0000","last_became_active":"2024-04-26T21:16:03.280615+0000","last_became_peered":"2024-04-26T21:16:03.280615+0000","last_unstale":"2024-04-26T21:16:04.694488+0000","last_undegraded":"2024-04-26T21:16:04.694488+0000","last_fullsized":"2024-04-26T21:16:04.694488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:32.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990112+0000","last_change":"2024-04-26T21:16:03.701218+0000","last_active":"2024-04-26T21:16:04.990112+0000","last_peered":"2024-04-26T21:16:04.990112+0000","last_clean":"2024-04-26T21:16:04.990112+0000","last_became_active":"2024-04-26T21:16:03.700895+0000","last_became_peered":"2024-04-26T21:16:03.700895+0000","last_unstale":"2024-04-26T21:16:04.990112+0000","last_undegraded":"2024-04-26T21:16:04.990112+0000","last_fullsized":"2024-04-26T21:16:04.990112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:44.104683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990519+0000","last_change":"2024-04-26T21:16:03.277235+0000","last_active":"2024-04-26T21:16:04.990519+0000","last_peered":"2024-04-26T21:16:04.990519+0000","last_clean":"2024-04-26T21:16:04.990519+0000","last_became_active":"2024-04-26T21:16:03.276901+0000","last_became_peered":"2024-04-26T21:16:03.276901+0000","last_unstale":"2024-04-26T21:16:04.990519+0000","last_undegraded":"2024-04-26T21:16:04.990519+0000","last_fullsized":"2024-04-26T21:16:04.990519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:16.297883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178787+0000","last_change":"2024-04-26T21:16:03.704318+0000","last_active":"2024-04-26T21:16:05.178787+0000","last_peered":"2024-04-26T21:16:05.178787+0000","last_clean":"2024-04-26T21:16:05.178787+0000","last_became_active":"2024-04-26T21:16:03.703907+0000","last_became_peered":"2024-04-26T21:16:03.703907+0000","last_unstale":"2024-04-26T21:16:05.178787+0000","last_undegraded":"2024-04-26T21:16:05.178787+0000","last_fullsized":"2024-04-26T21:16:05.178787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:16.431246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691082+0000","last_change":"2024-04-26T21:16:03.283158+0000","last_active":"2024-04-26T21:16:04.691082+0000","last_peered":"2024-04-26T21:16:04.691082+0000","last_clean":"2024-04-26T21:16:04.691082+0000","last_became_active":"2024-04-26T21:16:03.282838+0000","last_became_peered":"2024-04-26T21:16:03.282838+0000","last_unstale":"2024-04-26T21:16:04.691082+0000","last_undegraded":"2024-04-26T21:16:04.691082+0000","last_fullsized":"2024-04-26T21:16:04.691082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:48.567599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990976+0000","last_change":"2024-04-26T21:16:03.705026+0000","last_active":"2024-04-26T21:16:04.990976+0000","last_peered":"2024-04-26T21:16:04.990976+0000","last_clean":"2024-04-26T21:16:04.990976+0000","last_became_active":"2024-04-26T21:16:03.703336+0000","last_became_peered":"2024-04-26T21:16:03.703336+0000","last_unstale":"2024-04-26T21:16:04.990976+0000","last_undegraded":"2024-04-26T21:16:04.990976+0000","last_fullsized":"2024-04-26T21:16:04.990976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:11.746362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990195+0000","last_change":"2024-04-26T21:16:03.279263+0000","last_active":"2024-04-26T21:16:04.990195+0000","last_peered":"2024-04-26T21:16:04.990195+0000","last_clean":"2024-04-26T21:16:04.990195+0000","last_became_active":"2024-04-26T21:16:03.278948+0000","last_became_peered":"2024-04-26T21:16:03.278948+0000","last_unstale":"2024-04-26T21:16:04.990195+0000","last_undegraded":"2024-04-26T21:16:04.990195+0000","last_fullsized":"2024-04-26T21:16:04.990195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:39.078346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690780+0000","last_change":"2024-04-26T21:16:03.702913+0000","last_active":"2024-04-26T21:16:04.690780+0000","last_peered":"2024-04-26T21:16:04.690780+0000","last_clean":"2024-04-26T21:16:04.690780+0000","last_became_active":"2024-04-26T21:16:03.702507+0000","last_became_peered":"2024-04-26T21:16:03.702507+0000","last_unstale":"2024-04-26T21:16:04.690780+0000","last_undegraded":"2024-04-26T21:16:04.690780+0000","last_fullsized":"2024-04-26T21:16:04.690780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:51.846983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691372+0000","last_change":"2024-04-26T21:16:03.281898+0000","last_active":"2024-04-26T21:16:04.691372+0000","last_peered":"2024-04-26T21:16:04.691372+0000","last_clean":"2024-04-26T21:16:04.691372+0000","last_became_active":"2024-04-26T21:16:03.281544+0000","last_became_peered":"2024-04-26T21:16:03.281544+0000","last_unstale":"2024-04-26T21:16:04.691372+0000","last_undegraded":"2024-04-26T21:16:04.691372+0000","last_fullsized":"2024-04-26T21:16:04.691372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:39.061050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690605+0000","last_change":"2024-04-26T21:16:03.703006+0000","last_active":"2024-04-26T21:16:04.690605+0000","last_peered":"2024-04-26T21:16:04.690605+0000","last_clean":"2024-04-26T21:16:04.690605+0000","last_became_active":"2024-04-26T21:16:03.702705+0000","last_became_peered":"2024-04-26T21:16:03.702705+0000","last_unstale":"2024-04-26T21:16:04.690605+0000","last_undegraded":"2024-04-26T21:16:04.690605+0000","last_fullsized":"2024-04-26T21:16:04.690605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:11.885993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689876+0000","last_change":"2024-04-26T21:16:03.280891+0000","last_active":"2024-04-26T21:16:04.689876+0000","last_peered":"2024-04-26T21:16:04.689876+0000","last_clean":"2024-04-26T21:16:04.689876+0000","last_became_active":"2024-04-26T21:16:03.280577+0000","last_became_peered":"2024-04-26T21:16:03.280577+0000","last_unstale":"2024-04-26T21:16:04.689876+0000","last_undegraded":"2024-04-26T21:16:04.689876+0000","last_fullsized":"2024-04-26T21:16:04.689876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:28.698997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690433+0000","last_change":"2024-04-26T21:16:03.704084+0000","last_active":"2024-04-26T21:16:04.690433+0000","last_peered":"2024-04-26T21:16:04.690433+0000","last_clean":"2024-04-26T21:16:04.690433+0000","last_became_active":"2024-04-26T21:16:03.703142+0000","last_became_peered":"2024-04-26T21:16:03.703142+0000","last_unstale":"2024-04-26T21:16:04.690433+0000","last_undegraded":"2024-04-26T21:16:04.690433+0000","last_fullsized":"2024-04-26T21:16:04.690433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:40.728179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.688369+0000","last_change":"2024-04-26T21:16:03.276651+0000","last_active":"2024-04-26T21:16:04.688369+0000","last_peered":"2024-04-26T21:16:04.688369+0000","last_clean":"2024-04-26T21:16:04.688369+0000","last_became_active":"2024-04-26T21:16:03.276347+0000","last_became_peered":"2024-04-26T21:16:03.276347+0000","last_unstale":"2024-04-26T21:16:04.688369+0000","last_undegraded":"2024-04-26T21:16:04.688369+0000","last_fullsized":"2024-04-26T21:16:04.688369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:48.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690718+0000","last_change":"2024-04-26T21:16:03.705647+0000","last_active":"2024-04-26T21:16:04.690718+0000","last_peered":"2024-04-26T21:16:04.690718+0000","last_clean":"2024-04-26T21:16:04.690718+0000","last_became_active":"2024-04-26T21:16:03.705251+0000","last_became_peered":"2024-04-26T21:16:03.705251+0000","last_unstale":"2024-04-26T21:16:04.690718+0000","last_undegraded":"2024-04-26T21:16:04.690718+0000","last_fullsized":"2024-04-26T21:16:04.690718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:17.427221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178756+0000","last_change":"2024-04-26T21:16:03.701759+0000","last_active":"2024-04-26T21:16:05.178756+0000","last_peered":"2024-04-26T21:16:05.178756+0000","last_clean":"2024-04-26T21:16:05.178756+0000","last_became_active":"2024-04-26T21:16:03.701382+0000","last_became_peered":"2024-04-26T21:16:03.701382+0000","last_unstale":"2024-04-26T21:16:05.178756+0000","last_undegraded":"2024-04-26T21:16:05.178756+0000","last_fullsized":"2024-04-26T21:16:05.178756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.789455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691185+0000","last_change":"2024-04-26T21:16:03.700239+0000","last_active":"2024-04-26T21:16:04.691185+0000","last_peered":"2024-04-26T21:16:04.691185+0000","last_clean":"2024-04-26T21:16:04.691185+0000","last_became_active":"2024-04-26T21:16:03.699957+0000","last_became_peered":"2024-04-26T21:16:03.699957+0000","last_unstale":"2024-04-26T21:16:04.691185+0000","last_undegraded":"2024-04-26T21:16:04.691185+0000","last_fullsized":"2024-04-26T21:16:04.691185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:40.744961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691102+0000","last_change":"2024-04-26T21:16:03.702012+0000","last_active":"2024-04-26T21:16:04.691102+0000","last_peered":"2024-04-26T21:16:04.691102+0000","last_clean":"2024-04-26T21:16:04.691102+0000","last_became_active":"2024-04-26T21:16:03.701066+0000","last_became_peered":"2024-04-26T21:16:03.701066+0000","last_unstale":"2024-04-26T21:16:04.691102+0000","last_undegraded":"2024-04-26T21:16:04.691102+0000","last_fullsized":"2024-04-26T21:16:04.691102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:45.337320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991591+0000","last_change":"2024-04-26T21:16:03.702640+0000","last_active":"2024-04-26T21:16:04.991591+0000","last_peered":"2024-04-26T21:16:04.991591+0000","last_clean":"2024-04-26T21:16:04.991591+0000","last_became_active":"2024-04-26T21:16:03.702339+0000","last_became_peered":"2024-04-26T21:16:03.702339+0000","last_unstale":"2024-04-26T21:16:04.991591+0000","last_undegraded":"2024-04-26T21:16:04.991591+0000","last_fullsized":"2024-04-26T21:16:04.991591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:24.083734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.688146+0000","last_change":"2024-04-26T21:16:03.708369+0000","last_active":"2024-04-26T21:16:04.688146+0000","last_peered":"2024-04-26T21:16:04.688146+0000","last_clean":"2024-04-26T21:16:04.688146+0000","last_became_active":"2024-04-26T21:16:03.708050+0000","last_became_peered":"2024-04-26T21:16:03.708050+0000","last_unstale":"2024-04-26T21:16:04.688146+0000","last_undegraded":"2024-04-26T21:16:04.688146+0000","last_fullsized":"2024-04-26T21:16:04.688146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:58.200043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178943+0000","last_change":"2024-04-26T21:16:03.700362+0000","last_active":"2024-04-26T21:16:05.178943+0000","last_peered":"2024-04-26T21:16:05.178943+0000","last_clean":"2024-04-26T21:16:05.178943+0000","last_became_active":"2024-04-26T21:16:03.700053+0000","last_became_peered":"2024-04-26T21:16:03.700053+0000","last_unstale":"2024-04-26T21:16:05.178943+0000","last_undegraded":"2024-04-26T21:16:05.178943+0000","last_fullsized":"2024-04-26T21:16:05.178943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:23.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691400+0000","last_change":"2024-04-26T21:16:03.707717+0000","last_active":"2024-04-26T21:16:04.691400+0000","last_peered":"2024-04-26T21:16:04.691400+0000","last_clean":"2024-04-26T21:16:04.691400+0000","last_became_active":"2024-04-26T21:16:03.707343+0000","last_became_peered":"2024-04-26T21:16:03.707343+0000","last_unstale":"2024-04-26T21:16:04.691400+0000","last_undegraded":"2024-04-26T21:16:04.691400+0000","last_fullsized":"2024-04-26T21:16:04.691400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:46.261231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991929+0000","last_change":"2024-04-26T21:16:03.702731+0000","last_active":"2024-04-26T21:16:04.991929+0000","last_peered":"2024-04-26T21:16:04.991929+0000","last_clean":"2024-04-26T21:16:04.991929+0000","last_became_active":"2024-04-26T21:16:03.702432+0000","last_became_peered":"2024-04-26T21:16:03.702432+0000","last_unstale":"2024-04-26T21:16:04.991929+0000","last_undegraded":"2024-04-26T21:16:04.991929+0000","last_fullsized":"2024-04-26T21:16:04.991929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:39.118876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689461+0000","last_change":"2024-04-26T21:16:03.699537+0000","last_active":"2024-04-26T21:16:04.689461+0000","last_peered":"2024-04-26T21:16:04.689461+0000","last_clean":"2024-04-26T21:16:04.689461+0000","last_became_active":"2024-04-26T21:16:03.699186+0000","last_became_peered":"2024-04-26T21:16:03.699186+0000","last_unstale":"2024-04-26T21:16:04.689461+0000","last_undegraded":"2024-04-26T21:16:04.689461+0000","last_fullsized":"2024-04-26T21:16:04.689461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:34.784929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990342+0000","last_change":"2024-04-26T21:16:03.709940+0000","last_active":"2024-04-26T21:16:04.990342+0000","last_peered":"2024-04-26T21:16:04.990342+0000","last_clean":"2024-04-26T21:16:04.990342+0000","last_became_active":"2024-04-26T21:16:03.707255+0000","last_became_peered":"2024-04-26T21:16:03.707255+0000","last_unstale":"2024-04-26T21:16:04.990342+0000","last_undegraded":"2024-04-26T21:16:04.990342+0000","last_fullsized":"2024-04-26T21:16:04.990342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:57.652194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694904+0000","last_change":"2024-04-26T21:16:03.711139+0000","last_active":"2024-04-26T21:16:04.694904+0000","last_peered":"2024-04-26T21:16:04.694904+0000","last_clean":"2024-04-26T21:16:04.694904+0000","last_became_active":"2024-04-26T21:16:03.709720+0000","last_became_peered":"2024-04-26T21:16:03.709720+0000","last_unstale":"2024-04-26T21:16:04.694904+0000","last_undegraded":"2024-04-26T21:16:04.694904+0000","last_fullsized":"2024-04-26T21:16:04.694904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:10.021914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.688104+0000","last_change":"2024-04-26T21:16:03.699140+0000","last_active":"2024-04-26T21:16:04.688104+0000","last_peered":"2024-04-26T21:16:04.688104+0000","last_clean":"2024-04-26T21:16:04.688104+0000","last_became_active":"2024-04-26T21:16:03.698824+0000","last_became_peered":"2024-04-26T21:16:03.698824+0000","last_unstale":"2024-04-26T21:16:04.688104+0000","last_undegraded":"2024-04-26T21:16:04.688104+0000","last_fullsized":"2024-04-26T21:16:04.688104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:55.896278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178690+0000","last_change":"2024-04-26T21:16:03.708258+0000","last_active":"2024-04-26T21:16:05.178690+0000","last_peered":"2024-04-26T21:16:05.178690+0000","last_clean":"2024-04-26T21:16:05.178690+0000","last_became_active":"2024-04-26T21:16:03.707932+0000","last_became_peered":"2024-04-26T21:16:03.707932+0000","last_unstale":"2024-04-26T21:16:05.178690+0000","last_undegraded":"2024-04-26T21:16:05.178690+0000","last_fullsized":"2024-04-26T21:16:05.178690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:56.758307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244196,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":43811,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":1033746,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":929792,"data_stored":442176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":974848,"data_stored":1032522,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":479799,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":441793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":1032122,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38227,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:16:11.382 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:16:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 -- 172.21.15.129:0/1520803751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 msgr2=0x7f7078102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 --2- 172.21.15.129:0/1520803751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 0x7f7078102da0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7068009ca0 tx=0x7f706802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 -- 172.21.15.129:0/1520803751 shutdown_connections 2024-04-26T21:16:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 --2- 172.21.15.129:0/1520803751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7078103b70 0x7f7078103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 --2- 172.21.15.129:0/1520803751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7078104530 0x7f707810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 --2- 172.21.15.129:0/1520803751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 0x7f7078102da0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 -- 172.21.15.129:0/1520803751 >> 172.21.15.129:0/1520803751 conn(0x7f70780fdf10 msgr2=0x7f7078100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 -- 172.21.15.129:0/1520803751 shutdown_connections 2024-04-26T21:16:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 -- 172.21.15.129:0/1520803751 wait complete. 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 Processor -- start 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 -- start start 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.516+0000 7f707dc20640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 0x7f7078198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f707dc20640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7078103b70 0x7f7078199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f707dc20640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7078104530 0x7f70781a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f707dc20640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f707810e480 con 0x7f7078102980 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f707dc20640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f707810e300 con 0x7f7078103b70 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f707dc20640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f707810e600 con 0x7f7078104530 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f70777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 0x7f7078198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f70777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 0x7f7078198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33594/0 (socket says 172.21.15.129:33594) 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f70777fe640 1 -- 172.21.15.129:0/1145826951 learned_addr learned my addr 172.21.15.129:0/1145826951 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f7076ffd640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7078103b70 0x7f7078199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f7077fff640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7078104530 0x7f70781a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f70777fe640 1 -- 172.21.15.129:0/1145826951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7078104530 msgr2=0x7f70781a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f70777fe640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7078104530 0x7f70781a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f70777fe640 1 -- 172.21.15.129:0/1145826951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7078103b70 msgr2=0x7f7078199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f70777fe640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7078103b70 0x7f7078199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f70777fe640 1 -- 172.21.15.129:0/1145826951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70781a0810 con 0x7f7078102980 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f70777fe640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 0x7f7078198bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f70680098d0 tx=0x7f7068031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f7074ff9640 1 -- 172.21.15.129:0/1145826951 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068003cb0 con 0x7f7078102980 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70781a0aa0 con 0x7f7078102980 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f7074ff9640 1 -- 172.21.15.129:0/1145826951 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7068003e50 con 0x7f7078102980 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f7074ff9640 1 -- 172.21.15.129:0/1145826951 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706802fc60 con 0x7f7078102980 2024-04-26T21:16:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.517+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70781a0ff0 con 0x7f7078102980 2024-04-26T21:16:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.518+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7078109d10 con 0x7f7078102980 2024-04-26T21:16:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.519+0000 7f7074ff9640 1 -- 172.21.15.129:0/1145826951 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7068004040 con 0x7f7078102980 2024-04-26T21:16:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.520+0000 7f7074ff9640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70640a3c40 0x7f70640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.520+0000 7f7076ffd640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70640a3c40 0x7f70640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.520+0000 7f7074ff9640 1 -- 172.21.15.129:0/1145826951 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f70680f1760 con 0x7f7078102980 2024-04-26T21:16:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.520+0000 7f7076ffd640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70640a3c40 0x7f70640a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f707819a100 tx=0x7f7060005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:11.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.524+0000 7f7074ff9640 1 -- 172.21.15.129:0/1145826951 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f70680be330 con 0x7f7078102980 2024-04-26T21:16:11.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.699+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7078062d70 con 0x7f70640a3c40 2024-04-26T21:16:11.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.708+0000 7f7074ff9640 1 -- 172.21.15.129:0/1145826951 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258833 (secure 0 0 0) 0x7f7078062d70 con 0x7f70640a3c40 2024-04-26T21:16:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:11.713 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-26T21:16:11.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70640a3c40 msgr2=0x7f70640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70640a3c40 0x7f70640a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f707819a100 tx=0x7f7060005f90 comp rx=0 tx=0).stop 2024-04-26T21:16:11.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 msgr2=0x7f7078198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:11.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 0x7f7078198bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f70680098d0 tx=0x7f7068031840 comp rx=0 tx=0).stop 2024-04-26T21:16:11.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 shutdown_connections 2024-04-26T21:16:11.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70640a3c40 0x7f70640a6100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7078103b70 0x7f7078199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7078104530 0x7f70781a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 --2- 172.21.15.129:0/1145826951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7078102980 0x7f7078198bf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:11.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 >> 172.21.15.129:0/1145826951 conn(0x7f70780fdf10 msgr2=0x7f7078106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:11.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 shutdown_connections 2024-04-26T21:16:11.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:11.711+0000 7f707dc20640 1 -- 172.21.15.129:0/1145826951 wait complete. 2024-04-26T21:16:11.723 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-26T21:16:11.276575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":134,"num_read_kb":116,"num_write":226,"num_write_kb":2952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":250,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360312,"kb_used_data":11076,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581512,"statfs":{"total":1151940427776,"available":1151571468288,"internally_reserved":0,"allocated":11341824,"data_stored":7113896,"data_compressed":108620,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001622"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.179037+0000","last_change":"2024-04-26T21:16:03.700581+0000","last_active":"2024-04-26T21:16:05.179037+0000","last_peered":"2024-04-26T21:16:05.179037+0000","last_clean":"2024-04-26T21:16:05.179037+0000","last_became_active":"2024-04-26T21:16:03.700280+0000","last_became_peered":"2024-04-26T21:16:03.700280+0000","last_unstale":"2024-04-26T21:16:05.179037+0000","last_undegraded":"2024-04-26T21:16:05.179037+0000","last_fullsized":"2024-04-26T21:16:05.179037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:01.329784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991539+0000","last_change":"2024-04-26T21:16:03.702058+0000","last_active":"2024-04-26T21:16:04.991539+0000","last_peered":"2024-04-26T21:16:04.991539+0000","last_clean":"2024-04-26T21:16:04.991539+0000","last_became_active":"2024-04-26T21:16:03.701728+0000","last_became_peered":"2024-04-26T21:16:03.701728+0000","last_unstale":"2024-04-26T21:16:04.991539+0000","last_undegraded":"2024-04-26T21:16:04.991539+0000","last_fullsized":"2024-04-26T21:16:04.991539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:12.718225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991160+0000","last_change":"2024-04-26T21:16:03.710461+0000","last_active":"2024-04-26T21:16:04.991160+0000","last_peered":"2024-04-26T21:16:04.991160+0000","last_clean":"2024-04-26T21:16:04.991160+0000","last_became_active":"2024-04-26T21:16:03.710130+0000","last_became_peered":"2024-04-26T21:16:03.710130+0000","last_unstale":"2024-04-26T21:16:04.991160+0000","last_undegraded":"2024-04-26T21:16:04.991160+0000","last_fullsized":"2024-04-26T21:16:04.991160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:21.294677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.905131+0000","last_change":"2024-04-26T21:16:03.706836+0000","last_active":"2024-04-26T21:16:04.905131+0000","last_peered":"2024-04-26T21:16:04.905131+0000","last_clean":"2024-04-26T21:16:04.905131+0000","last_became_active":"2024-04-26T21:16:03.706515+0000","last_became_peered":"2024-04-26T21:16:03.706515+0000","last_unstale":"2024-04-26T21:16:04.905131+0000","last_undegraded":"2024-04-26T21:16:04.905131+0000","last_fullsized":"2024-04-26T21:16:04.905131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:56.198337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990275+0000","last_change":"2024-04-26T21:16:03.704865+0000","last_active":"2024-04-26T21:16:04.990275+0000","last_peered":"2024-04-26T21:16:04.990275+0000","last_clean":"2024-04-26T21:16:04.990275+0000","last_became_active":"2024-04-26T21:16:03.704560+0000","last_became_peered":"2024-04-26T21:16:03.704560+0000","last_unstale":"2024-04-26T21:16:04.990275+0000","last_undegraded":"2024-04-26T21:16:04.990275+0000","last_fullsized":"2024-04-26T21:16:04.990275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:51:03.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692636+0000","last_change":"2024-04-26T21:16:03.700265+0000","last_active":"2024-04-26T21:16:04.692636+0000","last_peered":"2024-04-26T21:16:04.692636+0000","last_clean":"2024-04-26T21:16:04.692636+0000","last_became_active":"2024-04-26T21:16:03.699878+0000","last_became_peered":"2024-04-26T21:16:03.699878+0000","last_unstale":"2024-04-26T21:16:04.692636+0000","last_undegraded":"2024-04-26T21:16:04.692636+0000","last_fullsized":"2024-04-26T21:16:04.692636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:12.907254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690185+0000","last_change":"2024-04-26T21:16:03.697569+0000","last_active":"2024-04-26T21:16:04.690185+0000","last_peered":"2024-04-26T21:16:04.690185+0000","last_clean":"2024-04-26T21:16:04.690185+0000","last_became_active":"2024-04-26T21:16:03.697262+0000","last_became_peered":"2024-04-26T21:16:03.697262+0000","last_unstale":"2024-04-26T21:16:04.690185+0000","last_undegraded":"2024-04-26T21:16:04.690185+0000","last_fullsized":"2024-04-26T21:16:04.690185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:28.645750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990718+0000","last_change":"2024-04-26T21:16:03.705788+0000","last_active":"2024-04-26T21:16:04.990718+0000","last_peered":"2024-04-26T21:16:04.990718+0000","last_clean":"2024-04-26T21:16:04.990718+0000","last_became_active":"2024-04-26T21:16:03.705479+0000","last_became_peered":"2024-04-26T21:16:03.705479+0000","last_unstale":"2024-04-26T21:16:04.990718+0000","last_undegraded":"2024-04-26T21:16:04.990718+0000","last_fullsized":"2024-04-26T21:16:04.990718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:27.306777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.695162+0000","last_change":"2024-04-26T21:16:03.709716+0000","last_active":"2024-04-26T21:16:04.695162+0000","last_peered":"2024-04-26T21:16:04.695162+0000","last_clean":"2024-04-26T21:16:04.695162+0000","last_became_active":"2024-04-26T21:16:03.709219+0000","last_became_peered":"2024-04-26T21:16:03.709219+0000","last_unstale":"2024-04-26T21:16:04.695162+0000","last_undegraded":"2024-04-26T21:16:04.695162+0000","last_fullsized":"2024-04-26T21:16:04.695162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:18.936075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.693066+0000","last_change":"2024-04-26T21:16:03.694745+0000","last_active":"2024-04-26T21:16:04.693066+0000","last_peered":"2024-04-26T21:16:04.693066+0000","last_clean":"2024-04-26T21:16:04.693066+0000","last_became_active":"2024-04-26T21:16:03.694044+0000","last_became_peered":"2024-04-26T21:16:03.694044+0000","last_unstale":"2024-04-26T21:16:04.693066+0000","last_undegraded":"2024-04-26T21:16:04.693066+0000","last_fullsized":"2024-04-26T21:16:04.693066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:13.060467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690607+0000","last_change":"2024-04-26T21:16:03.706434+0000","last_active":"2024-04-26T21:16:04.690607+0000","last_peered":"2024-04-26T21:16:04.690607+0000","last_clean":"2024-04-26T21:16:04.690607+0000","last_became_active":"2024-04-26T21:16:03.706093+0000","last_became_peered":"2024-04-26T21:16:03.706093+0000","last_unstale":"2024-04-26T21:16:04.690607+0000","last_undegraded":"2024-04-26T21:16:04.690607+0000","last_fullsized":"2024-04-26T21:16:04.690607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:04.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.905135+0000","last_change":"2024-04-26T21:16:03.707672+0000","last_active":"2024-04-26T21:16:04.905135+0000","last_peered":"2024-04-26T21:16:04.905135+0000","last_clean":"2024-04-26T21:16:04.905135+0000","last_became_active":"2024-04-26T21:16:03.707344+0000","last_became_peered":"2024-04-26T21:16:03.707344+0000","last_unstale":"2024-04-26T21:16:04.905135+0000","last_undegraded":"2024-04-26T21:16:04.905135+0000","last_fullsized":"2024-04-26T21:16:04.905135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:07.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687931+0000","last_change":"2024-04-26T21:16:03.699816+0000","last_active":"2024-04-26T21:16:04.687931+0000","last_peered":"2024-04-26T21:16:04.687931+0000","last_clean":"2024-04-26T21:16:04.687931+0000","last_became_active":"2024-04-26T21:16:03.699514+0000","last_became_peered":"2024-04-26T21:16:03.699514+0000","last_unstale":"2024-04-26T21:16:04.687931+0000","last_undegraded":"2024-04-26T21:16:04.687931+0000","last_fullsized":"2024-04-26T21:16:04.687931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:59.230438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990746+0000","last_change":"2024-04-26T21:16:03.709627+0000","last_active":"2024-04-26T21:16:04.990746+0000","last_peered":"2024-04-26T21:16:04.990746+0000","last_clean":"2024-04-26T21:16:04.990746+0000","last_became_active":"2024-04-26T21:16:03.709238+0000","last_became_peered":"2024-04-26T21:16:03.709238+0000","last_unstale":"2024-04-26T21:16:04.990746+0000","last_undegraded":"2024-04-26T21:16:04.990746+0000","last_fullsized":"2024-04-26T21:16:04.990746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:55.303059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990402+0000","last_change":"2024-04-26T21:16:03.702275+0000","last_active":"2024-04-26T21:16:04.990402+0000","last_peered":"2024-04-26T21:16:04.990402+0000","last_clean":"2024-04-26T21:16:04.990402+0000","last_became_active":"2024-04-26T21:16:03.701966+0000","last_became_peered":"2024-04-26T21:16:03.701966+0000","last_unstale":"2024-04-26T21:16:04.990402+0000","last_undegraded":"2024-04-26T21:16:04.990402+0000","last_fullsized":"2024-04-26T21:16:04.990402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:02.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694686+0000","last_change":"2024-04-26T21:16:03.711979+0000","last_active":"2024-04-26T21:16:04.694686+0000","last_peered":"2024-04-26T21:16:04.694686+0000","last_clean":"2024-04-26T21:16:04.694686+0000","last_became_active":"2024-04-26T21:16:03.711637+0000","last_became_peered":"2024-04-26T21:16:03.711637+0000","last_unstale":"2024-04-26T21:16:04.694686+0000","last_undegraded":"2024-04-26T21:16:04.694686+0000","last_fullsized":"2024-04-26T21:16:04.694686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:27.817993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.179039+0000","last_change":"2024-04-26T21:16:03.701303+0000","last_active":"2024-04-26T21:16:05.179039+0000","last_peered":"2024-04-26T21:16:05.179039+0000","last_clean":"2024-04-26T21:16:05.179039+0000","last_became_active":"2024-04-26T21:16:03.701001+0000","last_became_peered":"2024-04-26T21:16:03.701001+0000","last_unstale":"2024-04-26T21:16:05.179039+0000","last_undegraded":"2024-04-26T21:16:05.179039+0000","last_fullsized":"2024-04-26T21:16:05.179039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:08.349915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990812+0000","last_change":"2024-04-26T21:16:03.696775+0000","last_active":"2024-04-26T21:16:04.990812+0000","last_peered":"2024-04-26T21:16:04.990812+0000","last_clean":"2024-04-26T21:16:04.990812+0000","last_became_active":"2024-04-26T21:16:03.696433+0000","last_became_peered":"2024-04-26T21:16:03.696433+0000","last_unstale":"2024-04-26T21:16:04.990812+0000","last_undegraded":"2024-04-26T21:16:04.990812+0000","last_fullsized":"2024-04-26T21:16:04.990812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:50.029169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691414+0000","last_change":"2024-04-26T21:16:03.698188+0000","last_active":"2024-04-26T21:16:04.691414+0000","last_peered":"2024-04-26T21:16:04.691414+0000","last_clean":"2024-04-26T21:16:04.691414+0000","last_became_active":"2024-04-26T21:16:03.697882+0000","last_became_peered":"2024-04-26T21:16:03.697882+0000","last_unstale":"2024-04-26T21:16:04.691414+0000","last_undegraded":"2024-04-26T21:16:04.691414+0000","last_fullsized":"2024-04-26T21:16:04.691414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:04.737619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990899+0000","last_change":"2024-04-26T21:16:03.275296+0000","last_active":"2024-04-26T21:16:04.990899+0000","last_peered":"2024-04-26T21:16:04.990899+0000","last_clean":"2024-04-26T21:16:04.990899+0000","last_became_active":"2024-04-26T21:16:03.274733+0000","last_became_peered":"2024-04-26T21:16:03.274733+0000","last_unstale":"2024-04-26T21:16:04.990899+0000","last_undegraded":"2024-04-26T21:16:04.990899+0000","last_fullsized":"2024-04-26T21:16:04.990899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:16.512934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991644+0000","last_change":"2024-04-26T21:16:03.705823+0000","last_active":"2024-04-26T21:16:04.991644+0000","last_peered":"2024-04-26T21:16:04.991644+0000","last_clean":"2024-04-26T21:16:04.991644+0000","last_became_active":"2024-04-26T21:16:03.705512+0000","last_became_peered":"2024-04-26T21:16:03.705512+0000","last_unstale":"2024-04-26T21:16:04.991644+0000","last_undegraded":"2024-04-26T21:16:04.991644+0000","last_fullsized":"2024-04-26T21:16:04.991644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:17.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991832+0000","last_change":"2024-04-26T21:16:03.285008+0000","last_active":"2024-04-26T21:16:04.991832+0000","last_peered":"2024-04-26T21:16:04.991832+0000","last_clean":"2024-04-26T21:16:04.991832+0000","last_became_active":"2024-04-26T21:16:03.284679+0000","last_became_peered":"2024-04-26T21:16:03.284679+0000","last_unstale":"2024-04-26T21:16:04.991832+0000","last_undegraded":"2024-04-26T21:16:04.991832+0000","last_fullsized":"2024-04-26T21:16:04.991832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:35.404086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.992144+0000","last_change":"2024-04-26T21:16:03.710571+0000","last_active":"2024-04-26T21:16:04.992144+0000","last_peered":"2024-04-26T21:16:04.992144+0000","last_clean":"2024-04-26T21:16:04.992144+0000","last_became_active":"2024-04-26T21:16:03.710237+0000","last_became_peered":"2024-04-26T21:16:03.710237+0000","last_unstale":"2024-04-26T21:16:04.992144+0000","last_undegraded":"2024-04-26T21:16:04.992144+0000","last_fullsized":"2024-04-26T21:16:04.992144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:53.188554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990535+0000","last_change":"2024-04-26T21:16:03.280177+0000","last_active":"2024-04-26T21:16:04.990535+0000","last_peered":"2024-04-26T21:16:04.990535+0000","last_clean":"2024-04-26T21:16:04.990535+0000","last_became_active":"2024-04-26T21:16:03.279881+0000","last_became_peered":"2024-04-26T21:16:03.279881+0000","last_unstale":"2024-04-26T21:16:04.990535+0000","last_undegraded":"2024-04-26T21:16:04.990535+0000","last_fullsized":"2024-04-26T21:16:04.990535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:41.489019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694801+0000","last_change":"2024-04-26T21:16:03.711355+0000","last_active":"2024-04-26T21:16:04.694801+0000","last_peered":"2024-04-26T21:16:04.694801+0000","last_clean":"2024-04-26T21:16:04.694801+0000","last_became_active":"2024-04-26T21:16:03.710885+0000","last_became_peered":"2024-04-26T21:16:03.710885+0000","last_unstale":"2024-04-26T21:16:04.694801+0000","last_undegraded":"2024-04-26T21:16:04.694801+0000","last_fullsized":"2024-04-26T21:16:04.694801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:10.515590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990429+0000","last_change":"2024-04-26T21:16:03.278484+0000","last_active":"2024-04-26T21:16:04.990429+0000","last_peered":"2024-04-26T21:16:04.990429+0000","last_clean":"2024-04-26T21:16:04.990429+0000","last_became_active":"2024-04-26T21:16:03.277402+0000","last_became_peered":"2024-04-26T21:16:03.277402+0000","last_unstale":"2024-04-26T21:16:04.990429+0000","last_undegraded":"2024-04-26T21:16:04.990429+0000","last_fullsized":"2024-04-26T21:16:04.990429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:21.807942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689994+0000","last_change":"2024-04-26T21:16:03.713577+0000","last_active":"2024-04-26T21:16:04.689994+0000","last_peered":"2024-04-26T21:16:04.689994+0000","last_clean":"2024-04-26T21:16:04.689994+0000","last_became_active":"2024-04-26T21:16:03.713318+0000","last_became_peered":"2024-04-26T21:16:03.713318+0000","last_unstale":"2024-04-26T21:16:04.689994+0000","last_undegraded":"2024-04-26T21:16:04.689994+0000","last_fullsized":"2024-04-26T21:16:04.689994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:57.888552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689718+0000","last_change":"2024-04-26T21:16:03.277219+0000","last_active":"2024-04-26T21:16:04.689718+0000","last_peered":"2024-04-26T21:16:04.689718+0000","last_clean":"2024-04-26T21:16:04.689718+0000","last_became_active":"2024-04-26T21:16:03.276885+0000","last_became_peered":"2024-04-26T21:16:03.276885+0000","last_unstale":"2024-04-26T21:16:04.689718+0000","last_undegraded":"2024-04-26T21:16:04.689718+0000","last_fullsized":"2024-04-26T21:16:04.689718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:45.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990316+0000","last_change":"2024-04-26T21:16:03.703453+0000","last_active":"2024-04-26T21:16:04.990316+0000","last_peered":"2024-04-26T21:16:04.990316+0000","last_clean":"2024-04-26T21:16:04.990316+0000","last_became_active":"2024-04-26T21:16:03.703161+0000","last_became_peered":"2024-04-26T21:16:03.703161+0000","last_unstale":"2024-04-26T21:16:04.990316+0000","last_undegraded":"2024-04-26T21:16:04.990316+0000","last_fullsized":"2024-04-26T21:16:04.990316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.999243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692475+0000","last_change":"2024-04-26T21:16:03.274815+0000","last_active":"2024-04-26T21:16:04.692475+0000","last_peered":"2024-04-26T21:16:04.692475+0000","last_clean":"2024-04-26T21:16:04.692475+0000","last_became_active":"2024-04-26T21:16:03.274509+0000","last_became_peered":"2024-04-26T21:16:03.274509+0000","last_unstale":"2024-04-26T21:16:04.692475+0000","last_undegraded":"2024-04-26T21:16:04.692475+0000","last_fullsized":"2024-04-26T21:16:04.692475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:27.212714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689700+0000","last_change":"2024-04-26T21:16:03.700986+0000","last_active":"2024-04-26T21:16:04.689700+0000","last_peered":"2024-04-26T21:16:04.689700+0000","last_clean":"2024-04-26T21:16:04.689700+0000","last_became_active":"2024-04-26T21:16:03.700630+0000","last_became_peered":"2024-04-26T21:16:03.700630+0000","last_unstale":"2024-04-26T21:16:04.689700+0000","last_undegraded":"2024-04-26T21:16:04.689700+0000","last_fullsized":"2024-04-26T21:16:04.689700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:12.703732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687693+0000","last_change":"2024-04-26T21:16:03.270417+0000","last_active":"2024-04-26T21:16:04.687693+0000","last_peered":"2024-04-26T21:16:04.687693+0000","last_clean":"2024-04-26T21:16:04.687693+0000","last_became_active":"2024-04-26T21:16:03.270066+0000","last_became_peered":"2024-04-26T21:16:03.270066+0000","last_unstale":"2024-04-26T21:16:04.687693+0000","last_undegraded":"2024-04-26T21:16:04.687693+0000","last_fullsized":"2024-04-26T21:16:04.687693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:46.794698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990564+0000","last_change":"2024-04-26T21:16:03.700221+0000","last_active":"2024-04-26T21:16:04.990564+0000","last_peered":"2024-04-26T21:16:04.990564+0000","last_clean":"2024-04-26T21:16:04.990564+0000","last_became_active":"2024-04-26T21:16:03.699798+0000","last_became_peered":"2024-04-26T21:16:03.699798+0000","last_unstale":"2024-04-26T21:16:04.990564+0000","last_undegraded":"2024-04-26T21:16:04.990564+0000","last_fullsized":"2024-04-26T21:16:04.990564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:52.083700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689324+0000","last_change":"2024-04-26T21:16:03.282355+0000","last_active":"2024-04-26T21:16:04.689324+0000","last_peered":"2024-04-26T21:16:04.689324+0000","last_clean":"2024-04-26T21:16:04.689324+0000","last_became_active":"2024-04-26T21:16:03.281169+0000","last_became_peered":"2024-04-26T21:16:03.281169+0000","last_unstale":"2024-04-26T21:16:04.689324+0000","last_undegraded":"2024-04-26T21:16:04.689324+0000","last_fullsized":"2024-04-26T21:16:04.689324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:55.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991575+0000","last_change":"2024-04-26T21:16:03.701016+0000","last_active":"2024-04-26T21:16:04.991575+0000","last_peered":"2024-04-26T21:16:04.991575+0000","last_clean":"2024-04-26T21:16:04.991575+0000","last_became_active":"2024-04-26T21:16:03.700711+0000","last_became_peered":"2024-04-26T21:16:03.700711+0000","last_unstale":"2024-04-26T21:16:04.991575+0000","last_undegraded":"2024-04-26T21:16:04.991575+0000","last_fullsized":"2024-04-26T21:16:04.991575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:13.390477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692871+0000","last_change":"2024-04-26T21:16:03.284948+0000","last_active":"2024-04-26T21:16:04.692871+0000","last_peered":"2024-04-26T21:16:04.692871+0000","last_clean":"2024-04-26T21:16:04.692871+0000","last_became_active":"2024-04-26T21:16:03.284570+0000","last_became_peered":"2024-04-26T21:16:03.284570+0000","last_unstale":"2024-04-26T21:16:04.692871+0000","last_undegraded":"2024-04-26T21:16:04.692871+0000","last_fullsized":"2024-04-26T21:16:04.692871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:22.883834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.695088+0000","last_change":"2024-04-26T21:16:03.710359+0000","last_active":"2024-04-26T21:16:04.695088+0000","last_peered":"2024-04-26T21:16:04.695088+0000","last_clean":"2024-04-26T21:16:04.695088+0000","last_became_active":"2024-04-26T21:16:03.710053+0000","last_became_peered":"2024-04-26T21:16:03.710053+0000","last_unstale":"2024-04-26T21:16:04.695088+0000","last_undegraded":"2024-04-26T21:16:04.695088+0000","last_fullsized":"2024-04-26T21:16:04.695088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:45.981083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690376+0000","last_change":"2024-04-26T21:16:03.278787+0000","last_active":"2024-04-26T21:16:04.690376+0000","last_peered":"2024-04-26T21:16:04.690376+0000","last_clean":"2024-04-26T21:16:04.690376+0000","last_became_active":"2024-04-26T21:16:03.278483+0000","last_became_peered":"2024-04-26T21:16:03.278483+0000","last_unstale":"2024-04-26T21:16:04.690376+0000","last_undegraded":"2024-04-26T21:16:04.690376+0000","last_fullsized":"2024-04-26T21:16:04.690376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:40.303372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.992068+0000","last_change":"2024-04-26T21:16:03.709501+0000","last_active":"2024-04-26T21:16:04.992068+0000","last_peered":"2024-04-26T21:16:04.992068+0000","last_clean":"2024-04-26T21:16:04.992068+0000","last_became_active":"2024-04-26T21:16:03.709118+0000","last_became_peered":"2024-04-26T21:16:03.709118+0000","last_unstale":"2024-04-26T21:16:04.992068+0000","last_undegraded":"2024-04-26T21:16:04.992068+0000","last_fullsized":"2024-04-26T21:16:04.992068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:23.904465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689984+0000","last_change":"2024-04-26T21:16:03.277387+0000","last_active":"2024-04-26T21:16:04.689984+0000","last_peered":"2024-04-26T21:16:04.689984+0000","last_clean":"2024-04-26T21:16:04.689984+0000","last_became_active":"2024-04-26T21:16:03.274765+0000","last_became_peered":"2024-04-26T21:16:03.274765+0000","last_unstale":"2024-04-26T21:16:04.689984+0000","last_undegraded":"2024-04-26T21:16:04.689984+0000","last_fullsized":"2024-04-26T21:16:04.689984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:51.081883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694930+0000","last_change":"2024-04-26T21:16:03.710977+0000","last_active":"2024-04-26T21:16:04.694930+0000","last_peered":"2024-04-26T21:16:04.694930+0000","last_clean":"2024-04-26T21:16:04.694930+0000","last_became_active":"2024-04-26T21:16:03.710284+0000","last_became_peered":"2024-04-26T21:16:03.710284+0000","last_unstale":"2024-04-26T21:16:04.694930+0000","last_undegraded":"2024-04-26T21:16:04.694930+0000","last_fullsized":"2024-04-26T21:16:04.694930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:46.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:07.680826+0000","last_change":"2024-04-26T21:16:03.276865+0000","last_active":"2024-04-26T21:16:07.680826+0000","last_peered":"2024-04-26T21:16:07.680826+0000","last_clean":"2024-04-26T21:16:07.680826+0000","last_became_active":"2024-04-26T21:16:03.276564+0000","last_became_peered":"2024-04-26T21:16:03.276564+0000","last_unstale":"2024-04-26T21:16:07.680826+0000","last_undegraded":"2024-04-26T21:16:07.680826+0000","last_fullsized":"2024-04-26T21:16:07.680826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:33.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990367+0000","last_change":"2024-04-26T21:16:03.700441+0000","last_active":"2024-04-26T21:16:04.990367+0000","last_peered":"2024-04-26T21:16:04.990367+0000","last_clean":"2024-04-26T21:16:04.990367+0000","last_became_active":"2024-04-26T21:16:03.700142+0000","last_became_peered":"2024-04-26T21:16:03.700142+0000","last_unstale":"2024-04-26T21:16:04.990367+0000","last_undegraded":"2024-04-26T21:16:04.990367+0000","last_fullsized":"2024-04-26T21:16:04.990367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:35.166603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:06.049054+0000","last_change":"2024-04-26T21:16:03.280144+0000","last_active":"2024-04-26T21:16:06.049054+0000","last_peered":"2024-04-26T21:16:06.049054+0000","last_clean":"2024-04-26T21:16:06.049054+0000","last_became_active":"2024-04-26T21:16:03.279844+0000","last_became_peered":"2024-04-26T21:16:03.279844+0000","last_unstale":"2024-04-26T21:16:06.049054+0000","last_undegraded":"2024-04-26T21:16:06.049054+0000","last_fullsized":"2024-04-26T21:16:06.049054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:00.704181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687987+0000","last_change":"2024-04-26T21:16:03.701732+0000","last_active":"2024-04-26T21:16:04.687987+0000","last_peered":"2024-04-26T21:16:04.687987+0000","last_clean":"2024-04-26T21:16:04.687987+0000","last_became_active":"2024-04-26T21:16:03.701424+0000","last_became_peered":"2024-04-26T21:16:03.701424+0000","last_unstale":"2024-04-26T21:16:04.687987+0000","last_undegraded":"2024-04-26T21:16:04.687987+0000","last_fullsized":"2024-04-26T21:16:04.687987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:02.293086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689250+0000","last_change":"2024-04-26T21:16:03.276816+0000","last_active":"2024-04-26T21:16:04.689250+0000","last_peered":"2024-04-26T21:16:04.689250+0000","last_clean":"2024-04-26T21:16:04.689250+0000","last_became_active":"2024-04-26T21:16:03.276471+0000","last_became_peered":"2024-04-26T21:16:03.276471+0000","last_unstale":"2024-04-26T21:16:04.689250+0000","last_undegraded":"2024-04-26T21:16:04.689250+0000","last_fullsized":"2024-04-26T21:16:04.689250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:44.630638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690511+0000","last_change":"2024-04-26T21:16:03.701002+0000","last_active":"2024-04-26T21:16:04.690511+0000","last_peered":"2024-04-26T21:16:04.690511+0000","last_clean":"2024-04-26T21:16:04.690511+0000","last_became_active":"2024-04-26T21:16:03.700649+0000","last_became_peered":"2024-04-26T21:16:03.700649+0000","last_unstale":"2024-04-26T21:16:04.690511+0000","last_undegraded":"2024-04-26T21:16:04.690511+0000","last_fullsized":"2024-04-26T21:16:04.690511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:45.571735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.904718+0000","last_change":"2024-04-26T21:16:03.273727+0000","last_active":"2024-04-26T21:16:04.904718+0000","last_peered":"2024-04-26T21:16:04.904718+0000","last_clean":"2024-04-26T21:16:04.904718+0000","last_became_active":"2024-04-26T21:16:03.273287+0000","last_became_peered":"2024-04-26T21:16:03.273287+0000","last_unstale":"2024-04-26T21:16:04.904718+0000","last_undegraded":"2024-04-26T21:16:04.904718+0000","last_fullsized":"2024-04-26T21:16:04.904718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:48.713646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690245+0000","last_change":"2024-04-26T21:16:03.696760+0000","last_active":"2024-04-26T21:16:04.690245+0000","last_peered":"2024-04-26T21:16:04.690245+0000","last_clean":"2024-04-26T21:16:04.690245+0000","last_became_active":"2024-04-26T21:16:03.696446+0000","last_became_peered":"2024-04-26T21:16:03.696446+0000","last_unstale":"2024-04-26T21:16:04.690245+0000","last_undegraded":"2024-04-26T21:16:04.690245+0000","last_fullsized":"2024-04-26T21:16:04.690245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:30.710360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990010+0000","last_change":"2024-04-26T21:16:03.282643+0000","last_active":"2024-04-26T21:16:04.990010+0000","last_peered":"2024-04-26T21:16:04.990010+0000","last_clean":"2024-04-26T21:16:04.990010+0000","last_became_active":"2024-04-26T21:16:03.282315+0000","last_became_peered":"2024-04-26T21:16:03.282315+0000","last_unstale":"2024-04-26T21:16:04.990010+0000","last_undegraded":"2024-04-26T21:16:04.990010+0000","last_fullsized":"2024-04-26T21:16:04.990010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:13.358002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689135+0000","last_change":"2024-04-26T21:16:03.711992+0000","last_active":"2024-04-26T21:16:04.689135+0000","last_peered":"2024-04-26T21:16:04.689135+0000","last_clean":"2024-04-26T21:16:04.689135+0000","last_became_active":"2024-04-26T21:16:03.711755+0000","last_became_peered":"2024-04-26T21:16:03.711755+0000","last_unstale":"2024-04-26T21:16:04.689135+0000","last_undegraded":"2024-04-26T21:16:04.689135+0000","last_fullsized":"2024-04-26T21:16:04.689135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:01.321530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990586+0000","last_change":"2024-04-26T21:16:03.274722+0000","last_active":"2024-04-26T21:16:04.990586+0000","last_peered":"2024-04-26T21:16:04.990586+0000","last_clean":"2024-04-26T21:16:04.990586+0000","last_became_active":"2024-04-26T21:16:03.274399+0000","last_became_peered":"2024-04-26T21:16:03.274399+0000","last_unstale":"2024-04-26T21:16:04.990586+0000","last_undegraded":"2024-04-26T21:16:04.990586+0000","last_fullsized":"2024-04-26T21:16:04.990586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:35.247456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178814+0000","last_change":"2024-04-26T21:16:03.698344+0000","last_active":"2024-04-26T21:16:05.178814+0000","last_peered":"2024-04-26T21:16:05.178814+0000","last_clean":"2024-04-26T21:16:05.178814+0000","last_became_active":"2024-04-26T21:16:03.698037+0000","last_became_peered":"2024-04-26T21:16:03.698037+0000","last_unstale":"2024-04-26T21:16:05.178814+0000","last_undegraded":"2024-04-26T21:16:05.178814+0000","last_fullsized":"2024-04-26T21:16:05.178814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.573421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990550+0000","last_change":"2024-04-26T21:16:03.276459+0000","last_active":"2024-04-26T21:16:04.990550+0000","last_peered":"2024-04-26T21:16:04.990550+0000","last_clean":"2024-04-26T21:16:04.990550+0000","last_became_active":"2024-04-26T21:16:03.276108+0000","last_became_peered":"2024-04-26T21:16:03.276108+0000","last_unstale":"2024-04-26T21:16:04.990550+0000","last_undegraded":"2024-04-26T21:16:04.990550+0000","last_fullsized":"2024-04-26T21:16:04.990550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.364925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692814+0000","last_change":"2024-04-26T21:16:03.696030+0000","last_active":"2024-04-26T21:16:04.692814+0000","last_peered":"2024-04-26T21:16:04.692814+0000","last_clean":"2024-04-26T21:16:04.692814+0000","last_became_active":"2024-04-26T21:16:03.695717+0000","last_became_peered":"2024-04-26T21:16:03.695717+0000","last_unstale":"2024-04-26T21:16:04.692814+0000","last_undegraded":"2024-04-26T21:16:04.692814+0000","last_fullsized":"2024-04-26T21:16:04.692814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:55.702216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690150+0000","last_change":"2024-04-26T21:16:03.269188+0000","last_active":"2024-04-26T21:16:04.690150+0000","last_peered":"2024-04-26T21:16:04.690150+0000","last_clean":"2024-04-26T21:16:04.690150+0000","last_became_active":"2024-04-26T21:16:03.268850+0000","last_became_peered":"2024-04-26T21:16:03.268850+0000","last_unstale":"2024-04-26T21:16:04.690150+0000","last_undegraded":"2024-04-26T21:16:04.690150+0000","last_fullsized":"2024-04-26T21:16:04.690150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:54.183612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.692731+0000","last_change":"2024-04-26T21:16:03.700284+0000","last_active":"2024-04-26T21:16:04.692731+0000","last_peered":"2024-04-26T21:16:04.692731+0000","last_clean":"2024-04-26T21:16:04.692731+0000","last_became_active":"2024-04-26T21:16:03.699965+0000","last_became_peered":"2024-04-26T21:16:03.699965+0000","last_unstale":"2024-04-26T21:16:04.692731+0000","last_undegraded":"2024-04-26T21:16:04.692731+0000","last_fullsized":"2024-04-26T21:16:04.692731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:40.547296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689760+0000","last_change":"2024-04-26T21:16:03.701370+0000","last_active":"2024-04-26T21:16:04.689760+0000","last_peered":"2024-04-26T21:16:04.689760+0000","last_clean":"2024-04-26T21:16:04.689760+0000","last_became_active":"2024-04-26T21:16:03.700585+0000","last_became_peered":"2024-04-26T21:16:03.700585+0000","last_unstale":"2024-04-26T21:16:04.689760+0000","last_undegraded":"2024-04-26T21:16:04.689760+0000","last_fullsized":"2024-04-26T21:16:04.689760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.348009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687467+0000","last_change":"2024-04-26T21:16:03.281141+0000","last_active":"2024-04-26T21:16:04.687467+0000","last_peered":"2024-04-26T21:16:04.687467+0000","last_clean":"2024-04-26T21:16:04.687467+0000","last_became_active":"2024-04-26T21:16:03.280812+0000","last_became_peered":"2024-04-26T21:16:03.280812+0000","last_unstale":"2024-04-26T21:16:04.687467+0000","last_undegraded":"2024-04-26T21:16:04.687467+0000","last_fullsized":"2024-04-26T21:16:04.687467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:11.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991138+0000","last_change":"2024-04-26T21:16:03.711312+0000","last_active":"2024-04-26T21:16:04.991138+0000","last_peered":"2024-04-26T21:16:04.991138+0000","last_clean":"2024-04-26T21:16:04.991138+0000","last_became_active":"2024-04-26T21:16:03.711026+0000","last_became_peered":"2024-04-26T21:16:03.711026+0000","last_unstale":"2024-04-26T21:16:04.991138+0000","last_undegraded":"2024-04-26T21:16:04.991138+0000","last_fullsized":"2024-04-26T21:16:04.991138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:10.185877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.904868+0000","last_change":"2024-04-26T21:16:03.273428+0000","last_active":"2024-04-26T21:16:04.904868+0000","last_peered":"2024-04-26T21:16:04.904868+0000","last_clean":"2024-04-26T21:16:04.904868+0000","last_became_active":"2024-04-26T21:16:03.272854+0000","last_became_peered":"2024-04-26T21:16:03.272854+0000","last_unstale":"2024-04-26T21:16:04.904868+0000","last_undegraded":"2024-04-26T21:16:04.904868+0000","last_fullsized":"2024-04-26T21:16:04.904868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:48.735059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":336,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690028+0000","last_change":"2024-04-26T21:14:50.729892+0000","last_active":"2024-04-26T21:16:04.690028+0000","last_peered":"2024-04-26T21:16:04.690028+0000","last_clean":"2024-04-26T21:16:04.690028+0000","last_became_active":"2024-04-26T21:14:50.729430+0000","last_became_peered":"2024-04-26T21:14:50.729430+0000","last_unstale":"2024-04-26T21:16:04.690028+0000","last_undegraded":"2024-04-26T21:16:04.690028+0000","last_fullsized":"2024-04-26T21:16:04.690028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_clean_scrub_stamp":"2024-04-26T21:11:22.039289+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:13.510971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687729+0000","last_change":"2024-04-26T21:16:03.700702+0000","last_active":"2024-04-26T21:16:04.687729+0000","last_peered":"2024-04-26T21:16:04.687729+0000","last_clean":"2024-04-26T21:16:04.687729+0000","last_became_active":"2024-04-26T21:16:03.700412+0000","last_became_peered":"2024-04-26T21:16:03.700412+0000","last_unstale":"2024-04-26T21:16:04.687729+0000","last_undegraded":"2024-04-26T21:16:04.687729+0000","last_fullsized":"2024-04-26T21:16:04.687729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:49.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690256+0000","last_change":"2024-04-26T21:16:03.271730+0000","last_active":"2024-04-26T21:16:04.690256+0000","last_peered":"2024-04-26T21:16:04.690256+0000","last_clean":"2024-04-26T21:16:04.690256+0000","last_became_active":"2024-04-26T21:16:03.271401+0000","last_became_peered":"2024-04-26T21:16:03.271401+0000","last_unstale":"2024-04-26T21:16:04.690256+0000","last_undegraded":"2024-04-26T21:16:04.690256+0000","last_fullsized":"2024-04-26T21:16:04.690256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:16.469022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990304+0000","last_change":"2024-04-26T21:16:03.276229+0000","last_active":"2024-04-26T21:16:04.990304+0000","last_peered":"2024-04-26T21:16:04.990304+0000","last_clean":"2024-04-26T21:16:04.990304+0000","last_became_active":"2024-04-26T21:16:03.275910+0000","last_became_peered":"2024-04-26T21:16:03.275910+0000","last_unstale":"2024-04-26T21:16:04.990304+0000","last_undegraded":"2024-04-26T21:16:04.990304+0000","last_fullsized":"2024-04-26T21:16:04.990304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:03.932254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.687898+0000","last_change":"2024-04-26T21:16:03.698293+0000","last_active":"2024-04-26T21:16:04.687898+0000","last_peered":"2024-04-26T21:16:04.687898+0000","last_clean":"2024-04-26T21:16:04.687898+0000","last_became_active":"2024-04-26T21:16:03.697935+0000","last_became_peered":"2024-04-26T21:16:03.697935+0000","last_unstale":"2024-04-26T21:16:04.687898+0000","last_undegraded":"2024-04-26T21:16:04.687898+0000","last_fullsized":"2024-04-26T21:16:04.687898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:54.292921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.682425+0000","last_change":"2024-04-26T21:16:03.275162+0000","last_active":"2024-04-26T21:16:05.682425+0000","last_peered":"2024-04-26T21:16:05.682425+0000","last_clean":"2024-04-26T21:16:05.682425+0000","last_became_active":"2024-04-26T21:16:03.274791+0000","last_became_peered":"2024-04-26T21:16:03.274791+0000","last_unstale":"2024-04-26T21:16:05.682425+0000","last_undegraded":"2024-04-26T21:16:05.682425+0000","last_fullsized":"2024-04-26T21:16:05.682425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:15.583747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.992098+0000","last_change":"2024-04-26T21:16:03.701473+0000","last_active":"2024-04-26T21:16:04.992098+0000","last_peered":"2024-04-26T21:16:04.992098+0000","last_clean":"2024-04-26T21:16:04.992098+0000","last_became_active":"2024-04-26T21:16:03.701070+0000","last_became_peered":"2024-04-26T21:16:03.701070+0000","last_unstale":"2024-04-26T21:16:04.992098+0000","last_undegraded":"2024-04-26T21:16:04.992098+0000","last_fullsized":"2024-04-26T21:16:04.992098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:25.373301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.905090+0000","last_change":"2024-04-26T21:16:03.271036+0000","last_active":"2024-04-26T21:16:04.905090+0000","last_peered":"2024-04-26T21:16:04.905090+0000","last_clean":"2024-04-26T21:16:04.905090+0000","last_became_active":"2024-04-26T21:16:03.270686+0000","last_became_peered":"2024-04-26T21:16:03.270686+0000","last_unstale":"2024-04-26T21:16:04.905090+0000","last_undegraded":"2024-04-26T21:16:04.905090+0000","last_fullsized":"2024-04-26T21:16:04.905090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:21.025757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991688+0000","last_change":"2024-04-26T21:16:03.705314+0000","last_active":"2024-04-26T21:16:04.991688+0000","last_peered":"2024-04-26T21:16:04.991688+0000","last_clean":"2024-04-26T21:16:04.991688+0000","last_became_active":"2024-04-26T21:16:03.704952+0000","last_became_peered":"2024-04-26T21:16:03.704952+0000","last_unstale":"2024-04-26T21:16:04.991688+0000","last_undegraded":"2024-04-26T21:16:04.991688+0000","last_fullsized":"2024-04-26T21:16:04.991688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:57.363909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694488+0000","last_change":"2024-04-26T21:16:03.280995+0000","last_active":"2024-04-26T21:16:04.694488+0000","last_peered":"2024-04-26T21:16:04.694488+0000","last_clean":"2024-04-26T21:16:04.694488+0000","last_became_active":"2024-04-26T21:16:03.280615+0000","last_became_peered":"2024-04-26T21:16:03.280615+0000","last_unstale":"2024-04-26T21:16:04.694488+0000","last_undegraded":"2024-04-26T21:16:04.694488+0000","last_fullsized":"2024-04-26T21:16:04.694488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:32.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990112+0000","last_change":"2024-04-26T21:16:03.701218+0000","last_active":"2024-04-26T21:16:04.990112+0000","last_peered":"2024-04-26T21:16:04.990112+0000","last_clean":"2024-04-26T21:16:04.990112+0000","last_became_active":"2024-04-26T21:16:03.700895+0000","last_became_peered":"2024-04-26T21:16:03.700895+0000","last_unstale":"2024-04-26T21:16:04.990112+0000","last_undegraded":"2024-04-26T21:16:04.990112+0000","last_fullsized":"2024-04-26T21:16:04.990112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:44.104683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990519+0000","last_change":"2024-04-26T21:16:03.277235+0000","last_active":"2024-04-26T21:16:04.990519+0000","last_peered":"2024-04-26T21:16:04.990519+0000","last_clean":"2024-04-26T21:16:04.990519+0000","last_became_active":"2024-04-26T21:16:03.276901+0000","last_became_peered":"2024-04-26T21:16:03.276901+0000","last_unstale":"2024-04-26T21:16:04.990519+0000","last_undegraded":"2024-04-26T21:16:04.990519+0000","last_fullsized":"2024-04-26T21:16:04.990519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:16.297883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178787+0000","last_change":"2024-04-26T21:16:03.704318+0000","last_active":"2024-04-26T21:16:05.178787+0000","last_peered":"2024-04-26T21:16:05.178787+0000","last_clean":"2024-04-26T21:16:05.178787+0000","last_became_active":"2024-04-26T21:16:03.703907+0000","last_became_peered":"2024-04-26T21:16:03.703907+0000","last_unstale":"2024-04-26T21:16:05.178787+0000","last_undegraded":"2024-04-26T21:16:05.178787+0000","last_fullsized":"2024-04-26T21:16:05.178787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:16.431246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691082+0000","last_change":"2024-04-26T21:16:03.283158+0000","last_active":"2024-04-26T21:16:04.691082+0000","last_peered":"2024-04-26T21:16:04.691082+0000","last_clean":"2024-04-26T21:16:04.691082+0000","last_became_active":"2024-04-26T21:16:03.282838+0000","last_became_peered":"2024-04-26T21:16:03.282838+0000","last_unstale":"2024-04-26T21:16:04.691082+0000","last_undegraded":"2024-04-26T21:16:04.691082+0000","last_fullsized":"2024-04-26T21:16:04.691082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:48.567599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990976+0000","last_change":"2024-04-26T21:16:03.705026+0000","last_active":"2024-04-26T21:16:04.990976+0000","last_peered":"2024-04-26T21:16:04.990976+0000","last_clean":"2024-04-26T21:16:04.990976+0000","last_became_active":"2024-04-26T21:16:03.703336+0000","last_became_peered":"2024-04-26T21:16:03.703336+0000","last_unstale":"2024-04-26T21:16:04.990976+0000","last_undegraded":"2024-04-26T21:16:04.990976+0000","last_fullsized":"2024-04-26T21:16:04.990976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:11.746362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990195+0000","last_change":"2024-04-26T21:16:03.279263+0000","last_active":"2024-04-26T21:16:04.990195+0000","last_peered":"2024-04-26T21:16:04.990195+0000","last_clean":"2024-04-26T21:16:04.990195+0000","last_became_active":"2024-04-26T21:16:03.278948+0000","last_became_peered":"2024-04-26T21:16:03.278948+0000","last_unstale":"2024-04-26T21:16:04.990195+0000","last_undegraded":"2024-04-26T21:16:04.990195+0000","last_fullsized":"2024-04-26T21:16:04.990195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:39.078346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690780+0000","last_change":"2024-04-26T21:16:03.702913+0000","last_active":"2024-04-26T21:16:04.690780+0000","last_peered":"2024-04-26T21:16:04.690780+0000","last_clean":"2024-04-26T21:16:04.690780+0000","last_became_active":"2024-04-26T21:16:03.702507+0000","last_became_peered":"2024-04-26T21:16:03.702507+0000","last_unstale":"2024-04-26T21:16:04.690780+0000","last_undegraded":"2024-04-26T21:16:04.690780+0000","last_fullsized":"2024-04-26T21:16:04.690780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:51.846983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691372+0000","last_change":"2024-04-26T21:16:03.281898+0000","last_active":"2024-04-26T21:16:04.691372+0000","last_peered":"2024-04-26T21:16:04.691372+0000","last_clean":"2024-04-26T21:16:04.691372+0000","last_became_active":"2024-04-26T21:16:03.281544+0000","last_became_peered":"2024-04-26T21:16:03.281544+0000","last_unstale":"2024-04-26T21:16:04.691372+0000","last_undegraded":"2024-04-26T21:16:04.691372+0000","last_fullsized":"2024-04-26T21:16:04.691372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:39.061050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690605+0000","last_change":"2024-04-26T21:16:03.703006+0000","last_active":"2024-04-26T21:16:04.690605+0000","last_peered":"2024-04-26T21:16:04.690605+0000","last_clean":"2024-04-26T21:16:04.690605+0000","last_became_active":"2024-04-26T21:16:03.702705+0000","last_became_peered":"2024-04-26T21:16:03.702705+0000","last_unstale":"2024-04-26T21:16:04.690605+0000","last_undegraded":"2024-04-26T21:16:04.690605+0000","last_fullsized":"2024-04-26T21:16:04.690605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:11.885993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689876+0000","last_change":"2024-04-26T21:16:03.280891+0000","last_active":"2024-04-26T21:16:04.689876+0000","last_peered":"2024-04-26T21:16:04.689876+0000","last_clean":"2024-04-26T21:16:04.689876+0000","last_became_active":"2024-04-26T21:16:03.280577+0000","last_became_peered":"2024-04-26T21:16:03.280577+0000","last_unstale":"2024-04-26T21:16:04.689876+0000","last_undegraded":"2024-04-26T21:16:04.689876+0000","last_fullsized":"2024-04-26T21:16:04.689876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:28.698997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690433+0000","last_change":"2024-04-26T21:16:03.704084+0000","last_active":"2024-04-26T21:16:04.690433+0000","last_peered":"2024-04-26T21:16:04.690433+0000","last_clean":"2024-04-26T21:16:04.690433+0000","last_became_active":"2024-04-26T21:16:03.703142+0000","last_became_peered":"2024-04-26T21:16:03.703142+0000","last_unstale":"2024-04-26T21:16:04.690433+0000","last_undegraded":"2024-04-26T21:16:04.690433+0000","last_fullsized":"2024-04-26T21:16:04.690433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:40.728179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.688369+0000","last_change":"2024-04-26T21:16:03.276651+0000","last_active":"2024-04-26T21:16:04.688369+0000","last_peered":"2024-04-26T21:16:04.688369+0000","last_clean":"2024-04-26T21:16:04.688369+0000","last_became_active":"2024-04-26T21:16:03.276347+0000","last_became_peered":"2024-04-26T21:16:03.276347+0000","last_unstale":"2024-04-26T21:16:04.688369+0000","last_undegraded":"2024-04-26T21:16:04.688369+0000","last_fullsized":"2024-04-26T21:16:04.688369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:48.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.690718+0000","last_change":"2024-04-26T21:16:03.705647+0000","last_active":"2024-04-26T21:16:04.690718+0000","last_peered":"2024-04-26T21:16:04.690718+0000","last_clean":"2024-04-26T21:16:04.690718+0000","last_became_active":"2024-04-26T21:16:03.705251+0000","last_became_peered":"2024-04-26T21:16:03.705251+0000","last_unstale":"2024-04-26T21:16:04.690718+0000","last_undegraded":"2024-04-26T21:16:04.690718+0000","last_fullsized":"2024-04-26T21:16:04.690718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:17.427221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178756+0000","last_change":"2024-04-26T21:16:03.701759+0000","last_active":"2024-04-26T21:16:05.178756+0000","last_peered":"2024-04-26T21:16:05.178756+0000","last_clean":"2024-04-26T21:16:05.178756+0000","last_became_active":"2024-04-26T21:16:03.701382+0000","last_became_peered":"2024-04-26T21:16:03.701382+0000","last_unstale":"2024-04-26T21:16:05.178756+0000","last_undegraded":"2024-04-26T21:16:05.178756+0000","last_fullsized":"2024-04-26T21:16:05.178756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.789455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691185+0000","last_change":"2024-04-26T21:16:03.700239+0000","last_active":"2024-04-26T21:16:04.691185+0000","last_peered":"2024-04-26T21:16:04.691185+0000","last_clean":"2024-04-26T21:16:04.691185+0000","last_became_active":"2024-04-26T21:16:03.699957+0000","last_became_peered":"2024-04-26T21:16:03.699957+0000","last_unstale":"2024-04-26T21:16:04.691185+0000","last_undegraded":"2024-04-26T21:16:04.691185+0000","last_fullsized":"2024-04-26T21:16:04.691185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:40.744961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691102+0000","last_change":"2024-04-26T21:16:03.702012+0000","last_active":"2024-04-26T21:16:04.691102+0000","last_peered":"2024-04-26T21:16:04.691102+0000","last_clean":"2024-04-26T21:16:04.691102+0000","last_became_active":"2024-04-26T21:16:03.701066+0000","last_became_peered":"2024-04-26T21:16:03.701066+0000","last_unstale":"2024-04-26T21:16:04.691102+0000","last_undegraded":"2024-04-26T21:16:04.691102+0000","last_fullsized":"2024-04-26T21:16:04.691102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:45.337320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991591+0000","last_change":"2024-04-26T21:16:03.702640+0000","last_active":"2024-04-26T21:16:04.991591+0000","last_peered":"2024-04-26T21:16:04.991591+0000","last_clean":"2024-04-26T21:16:04.991591+0000","last_became_active":"2024-04-26T21:16:03.702339+0000","last_became_peered":"2024-04-26T21:16:03.702339+0000","last_unstale":"2024-04-26T21:16:04.991591+0000","last_undegraded":"2024-04-26T21:16:04.991591+0000","last_fullsized":"2024-04-26T21:16:04.991591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:24.083734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.688146+0000","last_change":"2024-04-26T21:16:03.708369+0000","last_active":"2024-04-26T21:16:04.688146+0000","last_peered":"2024-04-26T21:16:04.688146+0000","last_clean":"2024-04-26T21:16:04.688146+0000","last_became_active":"2024-04-26T21:16:03.708050+0000","last_became_peered":"2024-04-26T21:16:03.708050+0000","last_unstale":"2024-04-26T21:16:04.688146+0000","last_undegraded":"2024-04-26T21:16:04.688146+0000","last_fullsized":"2024-04-26T21:16:04.688146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:58.200043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178943+0000","last_change":"2024-04-26T21:16:03.700362+0000","last_active":"2024-04-26T21:16:05.178943+0000","last_peered":"2024-04-26T21:16:05.178943+0000","last_clean":"2024-04-26T21:16:05.178943+0000","last_became_active":"2024-04-26T21:16:03.700053+0000","last_became_peered":"2024-04-26T21:16:03.700053+0000","last_unstale":"2024-04-26T21:16:05.178943+0000","last_undegraded":"2024-04-26T21:16:05.178943+0000","last_fullsized":"2024-04-26T21:16:05.178943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:23.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.691400+0000","last_change":"2024-04-26T21:16:03.707717+0000","last_active":"2024-04-26T21:16:04.691400+0000","last_peered":"2024-04-26T21:16:04.691400+0000","last_clean":"2024-04-26T21:16:04.691400+0000","last_became_active":"2024-04-26T21:16:03.707343+0000","last_became_peered":"2024-04-26T21:16:03.707343+0000","last_unstale":"2024-04-26T21:16:04.691400+0000","last_undegraded":"2024-04-26T21:16:04.691400+0000","last_fullsized":"2024-04-26T21:16:04.691400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:46.261231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.991929+0000","last_change":"2024-04-26T21:16:03.702731+0000","last_active":"2024-04-26T21:16:04.991929+0000","last_peered":"2024-04-26T21:16:04.991929+0000","last_clean":"2024-04-26T21:16:04.991929+0000","last_became_active":"2024-04-26T21:16:03.702432+0000","last_became_peered":"2024-04-26T21:16:03.702432+0000","last_unstale":"2024-04-26T21:16:04.991929+0000","last_undegraded":"2024-04-26T21:16:04.991929+0000","last_fullsized":"2024-04-26T21:16:04.991929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:39.118876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.689461+0000","last_change":"2024-04-26T21:16:03.699537+0000","last_active":"2024-04-26T21:16:04.689461+0000","last_peered":"2024-04-26T21:16:04.689461+0000","last_clean":"2024-04-26T21:16:04.689461+0000","last_became_active":"2024-04-26T21:16:03.699186+0000","last_became_peered":"2024-04-26T21:16:03.699186+0000","last_unstale":"2024-04-26T21:16:04.689461+0000","last_undegraded":"2024-04-26T21:16:04.689461+0000","last_fullsized":"2024-04-26T21:16:04.689461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:34.784929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.990342+0000","last_change":"2024-04-26T21:16:03.709940+0000","last_active":"2024-04-26T21:16:04.990342+0000","last_peered":"2024-04-26T21:16:04.990342+0000","last_clean":"2024-04-26T21:16:04.990342+0000","last_became_active":"2024-04-26T21:16:03.707255+0000","last_became_peered":"2024-04-26T21:16:03.707255+0000","last_unstale":"2024-04-26T21:16:04.990342+0000","last_undegraded":"2024-04-26T21:16:04.990342+0000","last_fullsized":"2024-04-26T21:16:04.990342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:57.652194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.694904+0000","last_change":"2024-04-26T21:16:03.711139+0000","last_active":"2024-04-26T21:16:04.694904+0000","last_peered":"2024-04-26T21:16:04.694904+0000","last_clean":"2024-04-26T21:16:04.694904+0000","last_became_active":"2024-04-26T21:16:03.709720+0000","last_became_peered":"2024-04-26T21:16:03.709720+0000","last_unstale":"2024-04-26T21:16:04.694904+0000","last_undegraded":"2024-04-26T21:16:04.694904+0000","last_fullsized":"2024-04-26T21:16:04.694904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:10.021914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:04.688104+0000","last_change":"2024-04-26T21:16:03.699140+0000","last_active":"2024-04-26T21:16:04.688104+0000","last_peered":"2024-04-26T21:16:04.688104+0000","last_clean":"2024-04-26T21:16:04.688104+0000","last_became_active":"2024-04-26T21:16:03.698824+0000","last_became_peered":"2024-04-26T21:16:03.698824+0000","last_unstale":"2024-04-26T21:16:04.688104+0000","last_undegraded":"2024-04-26T21:16:04.688104+0000","last_fullsized":"2024-04-26T21:16:04.688104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:55.896278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T21:16:05.178690+0000","last_change":"2024-04-26T21:16:03.708258+0000","last_active":"2024-04-26T21:16:05.178690+0000","last_peered":"2024-04-26T21:16:05.178690+0000","last_clean":"2024-04-26T21:16:05.178690+0000","last_became_active":"2024-04-26T21:16:03.707932+0000","last_became_peered":"2024-04-26T21:16:03.707932+0000","last_unstale":"2024-04-26T21:16:05.178690+0000","last_undegraded":"2024-04-26T21:16:05.178690+0000","last_fullsized":"2024-04-26T21:16:05.178690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:56.758307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244196,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":43811,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":1033746,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":929792,"data_stored":442176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":974848,"data_stored":1032522,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":479799,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":441793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":1032122,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38227,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:16:11.727 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:16:11.728 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T21:16:11.728 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T21:16:11.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:11 smithi129 ceph-mon[29237]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:11.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:11 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2592194901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:16:11.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:11 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1811534847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:11.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:11 smithi129 ceph-mon[29237]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T21:16:11.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:11 smithi129 ceph-mon[29237]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:16:11.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:11 smithi171 ceph-mon[33478]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:11.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:11 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2592194901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:16:11.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:11 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1811534847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:11.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:11 smithi171 ceph-mon[33478]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T21:16:11.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:11 smithi171 ceph-mon[33478]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:16:12.002 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:12.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:11 smithi150 ceph-mon[32290]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:16:12.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:11 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2592194901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:16:12.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:11 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1811534847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:12.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:11 smithi150 ceph-mon[32290]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T21:16:12.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:11 smithi150 ceph-mon[32290]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:16:12.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:12 smithi129 ceph-mon[29237]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:16:12.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:12 smithi171 ceph-mon[33478]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:16:13.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:12 smithi150 ceph-mon[32290]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:16:13.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.783+0000 7fcb19fe4700 1 -- 172.21.15.129:0/340066608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb141046a0 msgr2=0x7fcb14104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.783+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/340066608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb141046a0 0x7fcb14104b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb080099c0 tx=0x7fcb08009cd0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.783+0000 7fcb19fe4700 1 -- 172.21.15.129:0/340066608 shutdown_connections 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.783+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/340066608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb14105040 0x7fcb1410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.783+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/340066608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb141034a0 0x7fcb141038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.783+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/340066608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb141046a0 0x7fcb14104b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.783+0000 7fcb19fe4700 1 -- 172.21.15.129:0/340066608 >> 172.21.15.129:0/340066608 conn(0x7fcb140fea40 msgr2=0x7fcb14100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.783+0000 7fcb19fe4700 1 -- 172.21.15.129:0/340066608 shutdown_connections 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.783+0000 7fcb19fe4700 1 -- 172.21.15.129:0/340066608 wait complete. 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb19fe4700 1 Processor -- start 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb19fe4700 1 -- start start 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb19fe4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb141034a0 0x7fcb140771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb19fe4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb141046a0 0x7fcb14075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb19fe4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb14105040 0x7fcb14075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb19fe4700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1410f000 con 0x7fcb14105040 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb19fe4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1410ee80 con 0x7fcb141034a0 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb19fe4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1410f180 con 0x7fcb141046a0 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb14105040 0x7fcb14075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb14105040 0x7fcb14075d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36478/0 (socket says 172.21.15.129:36478) 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb13fff700 1 -- 172.21.15.129:0/702522556 learned_addr learned my addr 172.21.15.129:0/702522556 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.784+0000 7fcb12ffd700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb141046a0 0x7fcb14075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb137fe700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb141034a0 0x7fcb140771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb13fff700 1 -- 172.21.15.129:0/702522556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb141046a0 msgr2=0x7fcb14075810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb13fff700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb141046a0 0x7fcb14075810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb13fff700 1 -- 172.21.15.129:0/702522556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb141034a0 msgr2=0x7fcb140771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb13fff700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb141034a0 0x7fcb140771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb13fff700 1 -- 172.21.15.129:0/702522556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb14076600 con 0x7fcb14105040 2024-04-26T21:16:13.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb137fe700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb141034a0 0x7fcb140771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:13.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb13fff700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb14105040 0x7fcb14075d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00ada0 tx=0x7fcb0c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:13.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb10ff9700 1 -- 172.21.15.129:0/702522556 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c017020 con 0x7fcb14105040 2024-04-26T21:16:13.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb141a37a0 con 0x7fcb14105040 2024-04-26T21:16:13.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb10ff9700 1 -- 172.21.15.129:0/702522556 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb0c00ce30 con 0x7fcb14105040 2024-04-26T21:16:13.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb10ff9700 1 -- 172.21.15.129:0/702522556 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c005350 con 0x7fcb14105040 2024-04-26T21:16:13.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.785+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb141a3ce0 con 0x7fcb14105040 2024-04-26T21:16:13.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.786+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb14051960 con 0x7fcb14105040 2024-04-26T21:16:13.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.787+0000 7fcb10ff9700 1 -- 172.21.15.129:0/702522556 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fcb0c00c980 con 0x7fcb14105040 2024-04-26T21:16:13.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.788+0000 7fcb10ff9700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcafc0a3d80 0x7fcafc0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.788+0000 7fcb137fe700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcafc0a3d80 0x7fcafc0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.788+0000 7fcb10ff9700 1 -- 172.21.15.129:0/702522556 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fcb0c0cfab0 con 0x7fcb14105040 2024-04-26T21:16:13.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.788+0000 7fcb137fe700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcafc0a3d80 0x7fcafc0a6240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb14104500 tx=0x7fcb04008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:13.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:13.792+0000 7fcb10ff9700 1 -- 172.21.15.129:0/702522556 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcb0c021020 con 0x7fcb14105040 2024-04-26T21:16:13.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:13 smithi129 ceph-mon[29237]: pgmap v295: 97 pgs: 97 active+clean; 615 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-26T21:16:13.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:13 smithi171 ceph-mon[33478]: pgmap v295: 97 pgs: 97 active+clean; 615 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-26T21:16:14.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.072+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fcb14064030 con 0x7fcb14105040 2024-04-26T21:16:14.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.074+0000 7fcb10ff9700 1 -- 172.21.15.129:0/702522556 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fcb0c09c570 con 0x7fcb14105040 2024-04-26T21:16:14.076 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-04-26T21:16:14.076 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCeGSxmfNJlBBAAm1IPELAVwSNqvu6wQdakZw== 2024-04-26T21:16:14.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcafc0a3d80 msgr2=0x7fcafc0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:14.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcafc0a3d80 0x7fcafc0a6240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb14104500 tx=0x7fcb04008040 comp rx=0 tx=0).stop 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb14105040 msgr2=0x7fcb14075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb14105040 0x7fcb14075d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00ada0 tx=0x7fcb0c00c330 comp rx=0 tx=0).stop 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 shutdown_connections 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcafc0a3d80 0x7fcafc0a6240 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb14105040 0x7fcb14075d50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb141034a0 0x7fcb140771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 --2- 172.21.15.129:0/702522556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb141046a0 0x7fcb14075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 >> 172.21.15.129:0/702522556 conn(0x7fcb140fea40 msgr2=0x7fcb141079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 shutdown_connections 2024-04-26T21:16:14.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:14.077+0000 7fcb19fe4700 1 -- 172.21.15.129:0/702522556 wait complete. 2024-04-26T21:16:14.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:13 smithi150 ceph-mon[32290]: pgmap v295: 97 pgs: 97 active+clean; 615 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-26T21:16:14.874 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:14 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/702522556' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:16:14.874 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:14 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/702522556' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:16:14.905 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:16:14.905 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T21:16:14.905 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:16:14.942 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T21:16:14.943 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T21:16:14.943 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T21:16:14.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:14 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/702522556' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:16:14.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:14 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/702522556' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:16:15.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:14 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/702522556' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:16:15.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:14 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/702522556' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:16:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:15.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:15 smithi129 ceph-mon[29237]: pgmap v296: 97 pgs: 97 active+clean; 615 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2024-04-26T21:16:15.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:15 smithi171 ceph-mon[33478]: pgmap v296: 97 pgs: 97 active+clean; 615 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2024-04-26T21:16:16.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:15 smithi150 ceph-mon[32290]: pgmap v296: 97 pgs: 97 active+clean; 615 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2024-04-26T21:16:16.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.903+0000 7f961970f700 1 -- 172.21.15.129:0/1044560356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 msgr2=0x7f9614106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:16.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.903+0000 7f961970f700 1 --2- 172.21.15.129:0/1044560356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 0x7f9614106f80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f96080099c0 tx=0x7f9608009cd0 comp rx=0 tx=0).stop 2024-04-26T21:16:16.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.903+0000 7f961970f700 1 -- 172.21.15.129:0/1044560356 shutdown_connections 2024-04-26T21:16:16.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.903+0000 7f961970f700 1 --2- 172.21.15.129:0/1044560356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9614107660 0x7f961410f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:16.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.903+0000 7f961970f700 1 --2- 172.21.15.129:0/1044560356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9614102610 0x7f96141029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:16.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.903+0000 7f961970f700 1 --2- 172.21.15.129:0/1044560356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 0x7f9614106f80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:16.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.903+0000 7f961970f700 1 -- 172.21.15.129:0/1044560356 >> 172.21.15.129:0/1044560356 conn(0x7f9614078370 msgr2=0x7f9614078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:16.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.903+0000 7f961970f700 1 -- 172.21.15.129:0/1044560356 shutdown_connections 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.903+0000 7f961970f700 1 -- 172.21.15.129:0/1044560356 wait complete. 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.904+0000 7f961970f700 1 Processor -- start 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.904+0000 7f961970f700 1 -- start start 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.904+0000 7f961970f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9614102610 0x7f961419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.904+0000 7f961970f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 0x7f961419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.904+0000 7f961970f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9614107660 0x7f96141987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.904+0000 7f961970f700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9614112550 con 0x7f9614102fc0 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.904+0000 7f961970f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96141123d0 con 0x7f9614107660 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.904+0000 7f961970f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96141126d0 con 0x7f9614102610 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96127fc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 0x7f961419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96127fc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 0x7f961419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36496/0 (socket says 172.21.15.129:36496) 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96127fc700 1 -- 172.21.15.129:0/1541143096 learned_addr learned my addr 172.21.15.129:0/1541143096 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:16.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f9612ffd700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9614102610 0x7f961419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96137fe700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9614107660 0x7f96141987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96127fc700 1 -- 172.21.15.129:0/1541143096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9614102610 msgr2=0x7f961419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96127fc700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9614102610 0x7f961419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96127fc700 1 -- 172.21.15.129:0/1541143096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9614107660 msgr2=0x7f96141987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96127fc700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9614107660 0x7f96141987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96127fc700 1 -- 172.21.15.129:0/1541143096 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9614198fd0 con 0x7f9614102fc0 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f96127fc700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 0x7f961419ebb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9608009f50 tx=0x7f96080037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f95fbfff700 1 -- 172.21.15.129:0/1541143096 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608027070 con 0x7f9614102fc0 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9614199260 con 0x7f9614102fc0 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f95fbfff700 1 -- 172.21.15.129:0/1541143096 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9608003fa0 con 0x7f9614102fc0 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f95fbfff700 1 -- 172.21.15.129:0/1541143096 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608022d20 con 0x7f9614102fc0 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.905+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961406bb90 con 0x7f9614102fc0 2024-04-26T21:16:16.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.906+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9614103a60 con 0x7f9614102fc0 2024-04-26T21:16:16.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.907+0000 7f95fbfff700 1 -- 172.21.15.129:0/1541143096 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f960802a070 con 0x7f9614102fc0 2024-04-26T21:16:16.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.908+0000 7f95fbfff700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f95fc0a3cd0 0x7f95fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:16.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.908+0000 7f9612ffd700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f95fc0a3cd0 0x7f95fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:16.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.908+0000 7f95fbfff700 1 -- 172.21.15.129:0/1541143096 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f96080d19d0 con 0x7f9614102fc0 2024-04-26T21:16:16.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.908+0000 7f9612ffd700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f95fc0a3cd0 0x7f95fc0a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9604009a10 tx=0x7f9604009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:16.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:16.912+0000 7f95fbfff700 1 -- 172.21.15.129:0/1541143096 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9608031070 con 0x7f9614102fc0 2024-04-26T21:16:17.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.170+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9614199d20 con 0x7f9614102fc0 2024-04-26T21:16:17.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.175+0000 7f95fbfff700 1 -- 172.21.15.129:0/1541143096 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+265005 (secure 0 0 0) 0x7f9608013070 con 0x7f9614102fc0 2024-04-26T21:16:17.179 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:17.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f95fc0a3cd0 msgr2=0x7f95fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f95fc0a3cd0 0x7f95fc0a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9604009a10 tx=0x7f9604009190 comp rx=0 tx=0).stop 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 msgr2=0x7f961419ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 0x7f961419ebb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9608009f50 tx=0x7f96080037c0 comp rx=0 tx=0).stop 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 shutdown_connections 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f95fc0a3cd0 0x7f95fc0a6190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9614107660 0x7f96141987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9614102610 0x7f961419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 --2- 172.21.15.129:0/1541143096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9614102fc0 0x7f961419ebb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 >> 172.21.15.129:0/1541143096 conn(0x7f9614078370 msgr2=0x7f96141003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 shutdown_connections 2024-04-26T21:16:17.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:17.178+0000 7f961970f700 1 -- 172.21.15.129:0/1541143096 wait complete. 2024-04-26T21:16:17.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:17 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1541143096' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:16:17.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:17 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1541143096' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:16:17.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:17 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1541143096' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:16:17.973 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3856000307},{"type":"v1","addr":"172.21.15.129:6801","nonce":3856000307}]},"active_addr":"172.21.15.129:6801/3856000307","active_change":"2024-04-26T21:08:55.174206+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.129:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":1747341822}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":3246983948}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":991725485}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":2169219483}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":193023015}]}]} 2024-04-26T21:16:17.979 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T21:16:17.979 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T21:16:17.979 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:16:18.221 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:18.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:18 smithi171 ceph-mon[33478]: pgmap v297: 97 pgs: 97 active+clean; 615 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-26T21:16:18.516 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:18 smithi129 ceph-mon[29237]: pgmap v297: 97 pgs: 97 active+clean; 615 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-26T21:16:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:18 smithi150 ceph-mon[32290]: pgmap v297: 97 pgs: 97 active+clean; 615 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.048+0000 7efc6f6bf700 1 -- 172.21.15.129:0/45629057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68102fc0 msgr2=0x7efc68106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.048+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/45629057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68102fc0 0x7efc68106f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b0d0 tx=0x7efc5c00b3e0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.048+0000 7efc6f6bf700 1 -- 172.21.15.129:0/45629057 shutdown_connections 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.048+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/45629057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc68107660 0x7efc6810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.048+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/45629057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68102610 0x7efc681029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.048+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/45629057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68102fc0 0x7efc68106f80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.048+0000 7efc6f6bf700 1 -- 172.21.15.129:0/45629057 >> 172.21.15.129:0/45629057 conn(0x7efc68078370 msgr2=0x7efc68078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.048+0000 7efc6f6bf700 1 -- 172.21.15.129:0/45629057 shutdown_connections 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.048+0000 7efc6f6bf700 1 -- 172.21.15.129:0/45629057 wait complete. 2024-04-26T21:16:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6f6bf700 1 Processor -- start 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6f6bf700 1 -- start start 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6f6bf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68102610 0x7efc68104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6f6bf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc68102fc0 0x7efc68104560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6f6bf700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68107660 0x7efc68104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6f6bf700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc68112550 con 0x7efc68107660 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6f6bf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc681123d0 con 0x7efc68102fc0 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6f6bf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efc681126d0 con 0x7efc68102610 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6dc5c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68107660 0x7efc68104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6dc5c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68107660 0x7efc68104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36518/0 (socket says 172.21.15.129:36518) 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6dc5c700 1 -- 172.21.15.129:0/2156820382 learned_addr learned my addr 172.21.15.129:0/2156820382 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:20.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.049+0000 7efc6d45b700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68102610 0x7efc68104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc6cc5a700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc68102fc0 0x7efc68104560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc6dc5c700 1 -- 172.21.15.129:0/2156820382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68102610 msgr2=0x7efc68104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc6dc5c700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68102610 0x7efc68104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc6dc5c700 1 -- 172.21.15.129:0/2156820382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc68102fc0 msgr2=0x7efc68104560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc6dc5c700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc68102fc0 0x7efc68104560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc6dc5c700 1 -- 172.21.15.129:0/2156820382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc681a80d0 con 0x7efc68107660 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc6dc5c700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68107660 0x7efc68104aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc5800c970 tx=0x7efc5800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc667fc700 1 -- 172.21.15.129:0/2156820382 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004a30 con 0x7efc68107660 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc681a83c0 con 0x7efc68107660 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc667fc700 1 -- 172.21.15.129:0/2156820382 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc580054d0 con 0x7efc68107660 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc667fc700 1 -- 172.21.15.129:0/2156820382 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5800ec20 con 0x7efc68107660 2024-04-26T21:16:20.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.050+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc681a8880 con 0x7efc68107660 2024-04-26T21:16:20.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.051+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc68103a60 con 0x7efc68107660 2024-04-26T21:16:20.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.052+0000 7efc667fc700 1 -- 172.21.15.129:0/2156820382 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efc58005000 con 0x7efc68107660 2024-04-26T21:16:20.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.053+0000 7efc667fc700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efc500a3cd0 0x7efc500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:20.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.053+0000 7efc6d45b700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efc500a3cd0 0x7efc500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:20.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.053+0000 7efc667fc700 1 -- 172.21.15.129:0/2156820382 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7efc580cfe70 con 0x7efc68107660 2024-04-26T21:16:20.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.053+0000 7efc6d45b700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efc500a3cd0 0x7efc500a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efc60009a10 tx=0x7efc60009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:20.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.057+0000 7efc667fc700 1 -- 172.21.15.129:0/2156820382 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efc58014030 con 0x7efc68107660 2024-04-26T21:16:20.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.256+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc6802e490 con 0x7efc68107660 2024-04-26T21:16:20.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.258+0000 7efc667fc700 1 -- 172.21.15.129:0/2156820382 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22274 (secure 0 0 0) 0x7efc580052f0 con 0x7efc68107660 2024-04-26T21:16:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:20.259 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:04.683012+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:16:20.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efc500a3cd0 msgr2=0x7efc500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:20.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efc500a3cd0 0x7efc500a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efc60009a10 tx=0x7efc60009190 comp rx=0 tx=0).stop 2024-04-26T21:16:20.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68107660 msgr2=0x7efc68104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:20.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68107660 0x7efc68104aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc5800c970 tx=0x7efc5800cc80 comp rx=0 tx=0).stop 2024-04-26T21:16:20.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 shutdown_connections 2024-04-26T21:16:20.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efc500a3cd0 0x7efc500a6190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc68107660 0x7efc68104aa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68102610 0x7efc68104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 --2- 172.21.15.129:0/2156820382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc68102fc0 0x7efc68104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:20.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 >> 172.21.15.129:0/2156820382 conn(0x7efc68078370 msgr2=0x7efc6806be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:20.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 shutdown_connections 2024-04-26T21:16:20.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:20.260+0000 7efc6f6bf700 1 -- 172.21.15.129:0/2156820382 wait complete. 2024-04-26T21:16:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:20 smithi150 ceph-mon[32290]: pgmap v298: 97 pgs: 97 active+clean; 620 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-04-26T21:16:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:20 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2156820382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:20.698 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:20 smithi129 ceph-mon[29237]: pgmap v298: 97 pgs: 97 active+clean; 620 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-04-26T21:16:20.698 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:20 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2156820382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:20.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:20 smithi171 ceph-mon[33478]: pgmap v298: 97 pgs: 97 active+clean; 620 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-04-26T21:16:20.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:20 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2156820382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:21.008 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T21:16:21.008 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:16:21.245 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:22.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:22 smithi129 ceph-mon[29237]: pgmap v299: 97 pgs: 97 active+clean; 620 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-04-26T21:16:22.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:22 smithi150 ceph-mon[32290]: pgmap v299: 97 pgs: 97 active+clean; 620 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-04-26T21:16:22.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:22 smithi171 ceph-mon[33478]: pgmap v299: 97 pgs: 97 active+clean; 620 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-04-26T21:16:23.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 -- 172.21.15.129:0/3228107446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e41094a0 msgr2=0x7f33e4109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:23.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 --2- 172.21.15.129:0/3228107446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e41094a0 0x7f33e4109880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009990 tx=0x7f33d4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 -- 172.21.15.129:0/3228107446 shutdown_connections 2024-04-26T21:16:23.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 --2- 172.21.15.129:0/3228107446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e406c510 0x7f33e41102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 --2- 172.21.15.129:0/3228107446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e41094a0 0x7f33e4109880 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 --2- 172.21.15.129:0/3228107446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e406bb70 0x7f33e406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 -- 172.21.15.129:0/3228107446 >> 172.21.15.129:0/3228107446 conn(0x7f33e40792f0 msgr2=0x7f33e4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:23.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 -- 172.21.15.129:0/3228107446 shutdown_connections 2024-04-26T21:16:23.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 -- 172.21.15.129:0/3228107446 wait complete. 2024-04-26T21:16:23.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.075+0000 7f33e98aa700 1 Processor -- start 2024-04-26T21:16:23.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e98aa700 1 -- start start 2024-04-26T21:16:23.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e98aa700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e406bb70 0x7f33e419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:23.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e98aa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e406c510 0x7f33e419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:23.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e98aa700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e41094a0 0x7f33e41998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:23.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e98aa700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113590 con 0x7f33e406bb70 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e98aa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113410 con 0x7f33e406c510 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e98aa700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113710 con 0x7f33e41094a0 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e406bb70 0x7f33e419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e2ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e406bb70 0x7f33e419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58606/0 (socket says 172.21.15.129:58606) 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e2ffd700 1 -- 172.21.15.129:0/1129100894 learned_addr learned my addr 172.21.15.129:0/1129100894 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e27fc700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e406c510 0x7f33e419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e2ffd700 1 -- 172.21.15.129:0/1129100894 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e41094a0 msgr2=0x7f33e41998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e2ffd700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e41094a0 0x7f33e41998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e2ffd700 1 -- 172.21.15.129:0/1129100894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e406c510 msgr2=0x7f33e419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e2ffd700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e406c510 0x7f33e419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e2ffd700 1 -- 172.21.15.129:0/1129100894 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e419a010 con 0x7f33e406bb70 2024-04-26T21:16:23.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.076+0000 7f33e27fc700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e406c510 0x7f33e419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:23.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.077+0000 7f33e2ffd700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e406bb70 0x7f33e419f6f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f33d40057c0 tx=0x7f33d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:23.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.077+0000 7f33e88a8700 1 -- 172.21.15.129:0/1129100894 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4027070 con 0x7f33e406bb70 2024-04-26T21:16:23.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.077+0000 7f33e88a8700 1 -- 172.21.15.129:0/1129100894 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d4004020 con 0x7f33e406bb70 2024-04-26T21:16:23.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.077+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e419a2a0 con 0x7f33e406bb70 2024-04-26T21:16:23.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.077+0000 7f33e88a8700 1 -- 172.21.15.129:0/1129100894 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4022bf0 con 0x7f33e406bb70 2024-04-26T21:16:23.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.077+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41a65c0 con 0x7f33e406bb70 2024-04-26T21:16:23.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.078+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e406a110 con 0x7f33e406bb70 2024-04-26T21:16:23.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.079+0000 7f33e88a8700 1 -- 172.21.15.129:0/1129100894 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f33d402a070 con 0x7f33e406bb70 2024-04-26T21:16:23.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.079+0000 7f33e88a8700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33cc0a3cd0 0x7f33cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:23.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.080+0000 7f33e27fc700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33cc0a3cd0 0x7f33cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:23.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.080+0000 7f33e88a8700 1 -- 172.21.15.129:0/1129100894 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f33d40d12b0 con 0x7f33e406bb70 2024-04-26T21:16:23.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.080+0000 7f33e27fc700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33cc0a3cd0 0x7f33cc0a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33d8005fd0 tx=0x7f33d8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:23.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.084+0000 7f33e88a8700 1 -- 172.21.15.129:0/1129100894 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f33d405f060 con 0x7f33e406bb70 2024-04-26T21:16:23.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.290+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33e402e490 con 0x7f33e406bb70 2024-04-26T21:16:23.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.291+0000 7f33e88a8700 1 -- 172.21.15.129:0/1129100894 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22274 (secure 0 0 0) 0x7f33d4013070 con 0x7f33e406bb70 2024-04-26T21:16:23.293 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:16:23.293 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:04.683012+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.293+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33cc0a3cd0 msgr2=0x7f33cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.293+0000 7f33e98aa700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33cc0a3cd0 0x7f33cc0a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33d8005fd0 tx=0x7f33d8005e30 comp rx=0 tx=0).stop 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.293+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e406bb70 msgr2=0x7f33e419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.293+0000 7f33e98aa700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e406bb70 0x7f33e419f6f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f33d40057c0 tx=0x7f33d40057f0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.293+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 shutdown_connections 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.293+0000 7f33e98aa700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33cc0a3cd0 0x7f33cc0a6190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.293+0000 7f33e98aa700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e41094a0 0x7f33e41998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.293+0000 7f33e98aa700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33e406bb70 0x7f33e419f6f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.293+0000 7f33e98aa700 1 --2- 172.21.15.129:0/1129100894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e406c510 0x7f33e419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.294+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 >> 172.21.15.129:0/1129100894 conn(0x7f33e40792f0 msgr2=0x7f33e41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.294+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 shutdown_connections 2024-04-26T21:16:23.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:23.294+0000 7f33e98aa700 1 -- 172.21.15.129:0/1129100894 wait complete. 2024-04-26T21:16:23.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:23 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1129100894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:23.656 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:23 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1129100894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:23 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1129100894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:16:24.129 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T21:16:24.129 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T21:16:24.130 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T21:16:24.130 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T21:16:24.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T21:16:24.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T21:16:24.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T21:16:24.132 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T21:16:24.132 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T21:16:24.133 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T21:16:24.133 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T21:16:24.134 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T21:16:24.437 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:24 smithi129 ceph-mon[29237]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-26T21:16:24.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:24 smithi150 ceph-mon[32290]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-26T21:16:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:24 smithi171 ceph-mon[33478]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-26T21:16:24.825 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:24.825 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:24.838 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:24.839 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:24.847 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:24.847 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:26 smithi129 ceph-mon[29237]: pgmap v301: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-04-26T21:16:26.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:26 smithi150 ceph-mon[32290]: pgmap v301: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-04-26T21:16:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:26 smithi171 ceph-mon[33478]: pgmap v301: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-04-26T21:16:28.624 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:28 smithi129 ceph-mon[29237]: pgmap v302: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-04-26T21:16:28.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:28 smithi150 ceph-mon[32290]: pgmap v302: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-04-26T21:16:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:28 smithi171 ceph-mon[33478]: pgmap v302: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-04-26T21:16:30.380 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:30.400 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:30.401 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:30.580 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:30 smithi129 ceph-mon[29237]: pgmap v303: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-04-26T21:16:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:30 smithi150 ceph-mon[32290]: pgmap v303: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-04-26T21:16:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:30 smithi171 ceph-mon[33478]: pgmap v303: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-04-26T21:16:31.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.067+0000 7fdda8a61700 1 -- 172.21.15.129:0/2110153672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4103ea0 msgr2=0x7fdda41102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.067+0000 7fdda8a61700 1 --2- 172.21.15.129:0/2110153672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4103ea0 0x7fdda41102c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00b0a0 tx=0x7fdd9c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.067+0000 7fdda8a61700 1 -- 172.21.15.129:0/2110153672 shutdown_connections 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.067+0000 7fdda8a61700 1 --2- 172.21.15.129:0/2110153672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4103ea0 0x7fdda41102c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.067+0000 7fdda8a61700 1 --2- 172.21.15.129:0/2110153672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda4109480 0x7fdda4109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.067+0000 7fdda8a61700 1 --2- 172.21.15.129:0/2110153672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda41034e0 0x7fdda4103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.067+0000 7fdda8a61700 1 -- 172.21.15.129:0/2110153672 >> 172.21.15.129:0/2110153672 conn(0x7fdda407a5e0 msgr2=0x7fdda407a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.067+0000 7fdda8a61700 1 -- 172.21.15.129:0/2110153672 shutdown_connections 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.067+0000 7fdda8a61700 1 -- 172.21.15.129:0/2110153672 wait complete. 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.068+0000 7fdda8a61700 1 Processor -- start 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.068+0000 7fdda8a61700 1 -- start start 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.068+0000 7fdda8a61700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda41034e0 0x7fdda419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.068+0000 7fdda8a61700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4103ea0 0x7fdda419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:31.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.068+0000 7fdda8a61700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda4109480 0x7fdda4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.068+0000 7fdda8a61700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4113690 con 0x7fdda4103ea0 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.068+0000 7fdda8a61700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4113510 con 0x7fdda41034e0 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.068+0000 7fdda8a61700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4113810 con 0x7fdda4109480 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.068+0000 7fdda1d9b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4103ea0 0x7fdda419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda1d9b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4103ea0 0x7fdda419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58628/0 (socket says 172.21.15.129:58628) 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda1d9b700 1 -- 172.21.15.129:0/762520113 learned_addr learned my addr 172.21.15.129:0/762520113 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda2d9d700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda4109480 0x7fdda4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda259c700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda41034e0 0x7fdda419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda1d9b700 1 -- 172.21.15.129:0/762520113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda4109480 msgr2=0x7fdda4199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda1d9b700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda4109480 0x7fdda4199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda1d9b700 1 -- 172.21.15.129:0/762520113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda41034e0 msgr2=0x7fdda419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda1d9b700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda41034e0 0x7fdda419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda1d9b700 1 -- 172.21.15.129:0/762520113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda419a1b0 con 0x7fdda4103ea0 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda259c700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda41034e0 0x7fdda419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:31.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda2d9d700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda4109480 0x7fdda4199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda1d9b700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4103ea0 0x7fdda419fd80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400b530 tx=0x7fdd9400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdd937fe700 1 -- 172.21.15.129:0/762520113 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94015020 con 0x7fdda4103ea0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda419a4a0 con 0x7fdda4103ea0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdd937fe700 1 -- 172.21.15.129:0/762520113 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd94011ce0 con 0x7fdda4103ea0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdd937fe700 1 -- 172.21.15.129:0/762520113 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94010d20 con 0x7fdda4103ea0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.069+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda41a6710 con 0x7fdda4103ea0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.070+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdda4063bb0 con 0x7fdda4103ea0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.071+0000 7fdd937fe700 1 -- 172.21.15.129:0/762520113 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdd94018020 con 0x7fdda4103ea0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.072+0000 7fdd937fe700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdd8c0a3cd0 0x7fdd8c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.072+0000 7fdda259c700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdd8c0a3cd0 0x7fdd8c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.072+0000 7fdd937fe700 1 -- 172.21.15.129:0/762520113 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdd940d0270 con 0x7fdda4103ea0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.072+0000 7fdd937fe700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] conn(0x7fdd8c0ad4b0 0x7fdd8c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.072+0000 7fdd937fe700 1 -- 172.21.15.129:0/762520113 --> [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd94010ec0 con 0x7fdd8c0ad4b0 2024-04-26T21:16:31.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.072+0000 7fdd937fe700 1 -- 172.21.15.129:0/762520113 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fdd940d0750 con 0x7fdda4103ea0 2024-04-26T21:16:31.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.072+0000 7fdda2d9d700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] conn(0x7fdd8c0ad4b0 0x7fdd8c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:31.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.072+0000 7fdda259c700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdd8c0a3cd0 0x7fdd8c0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdda4103170 tx=0x7fdd98008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:31.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.073+0000 7fdda2d9d700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] conn(0x7fdd8c0ad4b0 0x7fdd8c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:31.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.073+0000 7fdd937fe700 1 -- 172.21.15.129:0/762520113 <== osd.6 v2:172.21.15.129:6818/3546434113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd94010ec0 con 0x7fdd8c0ad4b0 2024-04-26T21:16:31.082 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:31.082 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:31.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.097+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 --> [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdda4064030 con 0x7fdd8c0ad4b0 2024-04-26T21:16:31.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.098+0000 7fdd937fe700 1 -- 172.21.15.129:0/762520113 <== osd.6 v2:172.21.15.129:6818/3546434113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdda4064030 con 0x7fdd8c0ad4b0 2024-04-26T21:16:31.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.098+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] conn(0x7fdd8c0ad4b0 msgr2=0x7fdd8c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:31.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.098+0000 7fdda8a61700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] conn(0x7fdd8c0ad4b0 0x7fdd8c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.099+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdd8c0a3cd0 msgr2=0x7fdd8c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.099+0000 7fdda8a61700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdd8c0a3cd0 0x7fdd8c0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdda4103170 tx=0x7fdd98008040 comp rx=0 tx=0).stop 2024-04-26T21:16:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.099+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4103ea0 msgr2=0x7fdda419fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.099+0000 7fdda8a61700 1 --2- 172.21.15.129:0/762520113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4103ea0 0x7fdda419fd80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400b530 tx=0x7fdd9400b8f0 comp rx=0 tx=0).stop 2024-04-26T21:16:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.099+0000 7fdda2d9d700 1 -- 172.21.15.129:0/762520113 reap_dead start 2024-04-26T21:16:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.099+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 shutdown_connections 2024-04-26T21:16:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.099+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 >> 172.21.15.129:0/762520113 conn(0x7fdda407a5e0 msgr2=0x7fdda41015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.099+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 shutdown_connections 2024-04-26T21:16:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:31.099+0000 7fdda8a61700 1 -- 172.21.15.129:0/762520113 wait complete. 2024-04-26T21:16:32.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:32 smithi129 ceph-mon[29237]: pgmap v304: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-26T21:16:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:32 smithi150 ceph-mon[32290]: pgmap v304: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-26T21:16:32.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:32 smithi171 ceph-mon[33478]: pgmap v304: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-26T21:16:34.489 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:34 smithi129 ceph-mon[29237]: pgmap v305: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T21:16:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:34 smithi150 ceph-mon[32290]: pgmap v305: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T21:16:34.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:34 smithi171 ceph-mon[33478]: pgmap v305: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T21:16:36.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.933+0000 7f5ea48f5700 1 -- 172.21.15.129:0/1616612317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0103000 msgr2=0x7f5ea010f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:36.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.933+0000 7f5ea48f5700 1 --2- 172.21.15.129:0/1616612317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0103000 0x7f5ea010f180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800b0d0 tx=0x7f5e9800b3e0 comp rx=0 tx=0).stop 2024-04-26T21:16:36.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.933+0000 7f5ea48f5700 1 -- 172.21.15.129:0/1616612317 shutdown_connections 2024-04-26T21:16:36.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.933+0000 7f5ea48f5700 1 --2- 172.21.15.129:0/1616612317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0103000 0x7f5ea010f180 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:36.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.933+0000 7f5ea48f5700 1 --2- 172.21.15.129:0/1616612317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ea0108530 0x7f5ea0108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:36.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.933+0000 7f5ea48f5700 1 --2- 172.21.15.129:0/1616612317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea0102640 0x7f5ea0102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:36.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.933+0000 7f5ea48f5700 1 -- 172.21.15.129:0/1616612317 >> 172.21.15.129:0/1616612317 conn(0x7f5ea0079080 msgr2=0x7f5ea0079490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.933+0000 7f5ea48f5700 1 -- 172.21.15.129:0/1616612317 shutdown_connections 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.933+0000 7f5ea48f5700 1 -- 172.21.15.129:0/1616612317 wait complete. 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5ea48f5700 1 Processor -- start 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5ea48f5700 1 -- start start 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5ea48f5700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0102640 0x7f5ea019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5ea48f5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea0103000 0x7f5ea019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5ea48f5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ea0108530 0x7f5ea0199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5ea48f5700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea0112650 con 0x7f5ea0102640 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5ea48f5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01124d0 con 0x7f5ea0108530 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5ea48f5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01127d0 con 0x7f5ea0103000 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5e9e59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0102640 0x7f5ea019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5e9e59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0102640 0x7f5ea019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49300/0 (socket says 172.21.15.129:49300) 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5e9e59c700 1 -- 172.21.15.129:0/2876738281 learned_addr learned my addr 172.21.15.129:0/2876738281 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5e9dd9b700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea0103000 0x7f5ea019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:36.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5e9e59c700 1 -- 172.21.15.129:0/2876738281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea0103000 msgr2=0x7f5ea019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5e9e59c700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea0103000 0x7f5ea019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5e9e59c700 1 -- 172.21.15.129:0/2876738281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ea0108530 msgr2=0x7f5ea0199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5e9e59c700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ea0108530 0x7f5ea0199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.934+0000 7f5e9e59c700 1 -- 172.21.15.129:0/2876738281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea019a0b0 con 0x7f5ea0102640 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.935+0000 7f5e9e59c700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0102640 0x7f5ea019f790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000b530 tx=0x7f5e9000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.935+0000 7f5e8f7fe700 1 -- 172.21.15.129:0/2876738281 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90015020 con 0x7f5ea0102640 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.935+0000 7f5e8f7fe700 1 -- 172.21.15.129:0/2876738281 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e90011ce0 con 0x7f5ea0102640 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.935+0000 7f5e8f7fe700 1 -- 172.21.15.129:0/2876738281 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90010d20 con 0x7f5ea0102640 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.935+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea019a3a0 con 0x7f5ea0102640 2024-04-26T21:16:36.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.935+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea01a6660 con 0x7f5ea0102640 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.936+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ea0063bb0 con 0x7f5ea0102640 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.937+0000 7f5e8f7fe700 1 -- 172.21.15.129:0/2876738281 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5e90018020 con 0x7f5ea0102640 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.937+0000 7f5e8f7fe700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5e880a3cd0 0x7f5e880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.937+0000 7f5e9dd9b700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5e880a3cd0 0x7f5e880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.937+0000 7f5e8f7fe700 1 -- 172.21.15.129:0/2876738281 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5e900d0260 con 0x7f5ea0102640 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.938+0000 7f5e8f7fe700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] conn(0x7f5e880ad4b0 0x7f5e880af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.938+0000 7f5e8f7fe700 1 -- 172.21.15.129:0/2876738281 --> [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e90010ec0 con 0x7f5e880ad4b0 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.938+0000 7f5e8f7fe700 1 -- 172.21.15.129:0/2876738281 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5e900d0740 con 0x7f5ea0102640 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.938+0000 7f5e9dd9b700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5e880a3cd0 0x7f5e880a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5e94006fd0 tx=0x7f5e94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.938+0000 7f5e9ed9d700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] conn(0x7f5e880ad4b0 0x7f5e880af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.938+0000 7f5e9ed9d700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] conn(0x7f5e880ad4b0 0x7f5e880af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.939+0000 7f5e8f7fe700 1 -- 172.21.15.129:0/2876738281 <== osd.2 v2:172.21.15.171:6800/1157402243 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e90010ec0 con 0x7f5e880ad4b0 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.956+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 --> [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ea0064030 con 0x7f5e880ad4b0 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.957+0000 7f5e8f7fe700 1 -- 172.21.15.129:0/2876738281 <== osd.2 v2:172.21.15.171:6800/1157402243 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5ea0064030 con 0x7f5e880ad4b0 2024-04-26T21:16:36.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.957+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 >> [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] conn(0x7f5e880ad4b0 msgr2=0x7f5e880af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.957+0000 7f5ea48f5700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] conn(0x7f5e880ad4b0 0x7f5e880af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.958+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5e880a3cd0 msgr2=0x7f5e880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.958+0000 7f5ea48f5700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5e880a3cd0 0x7f5e880a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5e94006fd0 tx=0x7f5e94008040 comp rx=0 tx=0).stop 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.958+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0102640 msgr2=0x7f5ea019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.958+0000 7f5ea48f5700 1 --2- 172.21.15.129:0/2876738281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea0102640 0x7f5ea019f790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000b530 tx=0x7f5e9000b8f0 comp rx=0 tx=0).stop 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.958+0000 7f5e9ed9d700 1 -- 172.21.15.129:0/2876738281 reap_dead start 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.958+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 shutdown_connections 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.958+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 >> 172.21.15.129:0/2876738281 conn(0x7f5ea0079080 msgr2=0x7f5ea006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.958+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 shutdown_connections 2024-04-26T21:16:36.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:33.958+0000 7f5ea48f5700 1 -- 172.21.15.129:0/2876738281 wait complete. 2024-04-26T21:16:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:36 smithi150 ceph-mon[32290]: pgmap v306: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T21:16:36.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:36 smithi129 ceph-mon[29237]: pgmap v306: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T21:16:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:36 smithi171 ceph-mon[33478]: pgmap v306: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T21:16:37.221 INFO:teuthology.orchestra.run.smithi129.stdout:77309411394 2024-04-26T21:16:37.221 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T21:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:38 smithi150 ceph-mon[32290]: pgmap v307: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T21:16:38.640 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:38 smithi129 ceph-mon[29237]: pgmap v307: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T21:16:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:38 smithi171 ceph-mon[33478]: pgmap v307: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T21:16:38.967 INFO:teuthology.orchestra.run.smithi129.stdout:201863462957 2024-04-26T21:16:38.967 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T21:16:39.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.170+0000 7f9170bfe700 1 -- 172.21.15.129:0/1288534978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f916c107a90 msgr2=0x7f916c107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.170+0000 7f9170bfe700 1 --2- 172.21.15.129:0/1288534978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f916c107a90 0x7f916c107e70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f915c009990 tx=0x7f915c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.170+0000 7f9170bfe700 1 -- 172.21.15.129:0/1288534978 shutdown_connections 2024-04-26T21:16:39.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.170+0000 7f9170bfe700 1 --2- 172.21.15.129:0/1288534978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c1024a0 0x7f916c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.170+0000 7f9170bfe700 1 --2- 172.21.15.129:0/1288534978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f916c107a90 0x7f916c107e70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.170+0000 7f9170bfe700 1 --2- 172.21.15.129:0/1288534978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f916c101ae0 0x7f916c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.170+0000 7f9170bfe700 1 -- 172.21.15.129:0/1288534978 >> 172.21.15.129:0/1288534978 conn(0x7f916c078990 msgr2=0x7f916c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:39.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.170+0000 7f9170bfe700 1 -- 172.21.15.129:0/1288534978 shutdown_connections 2024-04-26T21:16:39.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.170+0000 7f9170bfe700 1 -- 172.21.15.129:0/1288534978 wait complete. 2024-04-26T21:16:39.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.171+0000 7f9170bfe700 1 Processor -- start 2024-04-26T21:16:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.171+0000 7f9170bfe700 1 -- start start 2024-04-26T21:16:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9170bfe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c101ae0 0x7f916c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9170bfe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f916c1024a0 0x7f916c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9170bfe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f916c107a90 0x7f916c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9170bfe700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f916c111c80 con 0x7f916c107a90 2024-04-26T21:16:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9170bfe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f916c111b00 con 0x7f916c1024a0 2024-04-26T21:16:39.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9170bfe700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f916c111e00 con 0x7f916c101ae0 2024-04-26T21:16:39.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f916ad9d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f916c107a90 0x7f916c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f916a59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c101ae0 0x7f916c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f916ad9d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f916c107a90 0x7f916c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49332/0 (socket says 172.21.15.129:49332) 2024-04-26T21:16:39.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f916ad9d700 1 -- 172.21.15.129:0/1980028886 learned_addr learned my addr 172.21.15.129:0/1980028886 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:39.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9169d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f916c1024a0 0x7f916c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9169d9b700 1 -- 172.21.15.129:0/1980028886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c101ae0 msgr2=0x7f916c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9169d9b700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c101ae0 0x7f916c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9169d9b700 1 -- 172.21.15.129:0/1980028886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f916c107a90 msgr2=0x7f916c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9169d9b700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f916c107a90 0x7f916c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.172+0000 7f9169d9b700 1 -- 172.21.15.129:0/1980028886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c198f80 con 0x7f916c1024a0 2024-04-26T21:16:39.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.173+0000 7f916ad9d700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f916c107a90 0x7f916c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:16:39.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.173+0000 7f9169d9b700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f916c1024a0 0x7f916c19eb60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f916000e9f0 tx=0x7f916000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:39.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.173+0000 7f915b7fe700 1 -- 172.21.15.129:0/1980028886 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916000c9c0 con 0x7f916c1024a0 2024-04-26T21:16:39.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.173+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c199270 con 0x7f916c1024a0 2024-04-26T21:16:39.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.173+0000 7f915b7fe700 1 -- 172.21.15.129:0/1980028886 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f916000cb60 con 0x7f916c1024a0 2024-04-26T21:16:39.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.173+0000 7f915b7fe700 1 -- 172.21.15.129:0/1980028886 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160018680 con 0x7f916c1024a0 2024-04-26T21:16:39.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.173+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c1a5480 con 0x7f916c1024a0 2024-04-26T21:16:39.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.174+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f916c1048d0 con 0x7f916c1024a0 2024-04-26T21:16:39.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.175+0000 7f915b7fe700 1 -- 172.21.15.129:0/1980028886 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9160010430 con 0x7f916c1024a0 2024-04-26T21:16:39.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.176+0000 7f915b7fe700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f91540a3cd0 0x7f91540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.176+0000 7f916a59c700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f91540a3cd0 0x7f91540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.176+0000 7f915b7fe700 1 -- 172.21.15.129:0/1980028886 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f91600d0bd0 con 0x7f916c1024a0 2024-04-26T21:16:39.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.176+0000 7f915b7fe700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] conn(0x7f91540ad4b0 0x7f91540af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.176+0000 7f915b7fe700 1 -- 172.21.15.129:0/1980028886 --> [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9160019ec0 con 0x7f91540ad4b0 2024-04-26T21:16:39.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.176+0000 7f915b7fe700 1 -- 172.21.15.129:0/1980028886 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f91600d0f80 con 0x7f916c1024a0 2024-04-26T21:16:39.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.176+0000 7f916a59c700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f91540a3cd0 0x7f91540a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f915c0098e0 tx=0x7f915c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:39.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.177+0000 7f916ad9d700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] conn(0x7f91540ad4b0 0x7f91540af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.177+0000 7f916ad9d700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] conn(0x7f91540ad4b0 0x7f91540af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:39.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.178+0000 7f915b7fe700 1 -- 172.21.15.129:0/1980028886 <== osd.4 v2:172.21.15.150:6808/155182298 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9160019ec0 con 0x7f91540ad4b0 2024-04-26T21:16:39.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.194+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 --> [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f916c1031d0 con 0x7f91540ad4b0 2024-04-26T21:16:39.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.194+0000 7f915b7fe700 1 -- 172.21.15.129:0/1980028886 <== osd.4 v2:172.21.15.150:6808/155182298 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f916c1031d0 con 0x7f91540ad4b0 2024-04-26T21:16:39.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 >> [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] conn(0x7f91540ad4b0 msgr2=0x7f91540af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] conn(0x7f91540ad4b0 0x7f91540af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f91540a3cd0 msgr2=0x7f91540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f91540a3cd0 0x7f91540a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f915c0098e0 tx=0x7f915c005960 comp rx=0 tx=0).stop 2024-04-26T21:16:39.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f916c1024a0 msgr2=0x7f916c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 --2- 172.21.15.129:0/1980028886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f916c1024a0 0x7f916c19eb60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f916000e9f0 tx=0x7f916000ed00 comp rx=0 tx=0).stop 2024-04-26T21:16:39.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f916ad9d700 1 -- 172.21.15.129:0/1980028886 reap_dead start 2024-04-26T21:16:39.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 shutdown_connections 2024-04-26T21:16:39.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 >> 172.21.15.129:0/1980028886 conn(0x7f916c078990 msgr2=0x7f916c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:39.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 shutdown_connections 2024-04-26T21:16:39.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.195+0000 7f9170bfe700 1 -- 172.21.15.129:0/1980028886 wait complete. 2024-04-26T21:16:39.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 -- 172.21.15.129:0/3831962083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1301024a0 msgr2=0x7fb13010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 --2- 172.21.15.129:0/3831962083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1301024a0 0x7fb13010e960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b0a0 tx=0x7fb12800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 -- 172.21.15.129:0/3831962083 shutdown_connections 2024-04-26T21:16:39.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 --2- 172.21.15.129:0/3831962083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1301024a0 0x7fb13010e960 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 --2- 172.21.15.129:0/3831962083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb130107a90 0x7fb130107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 --2- 172.21.15.129:0/3831962083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb130101ae0 0x7fb130101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 -- 172.21.15.129:0/3831962083 >> 172.21.15.129:0/3831962083 conn(0x7fb130078990 msgr2=0x7fb1300ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:39.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 -- 172.21.15.129:0/3831962083 shutdown_connections 2024-04-26T21:16:39.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 -- 172.21.15.129:0/3831962083 wait complete. 2024-04-26T21:16:39.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.631+0000 7fb13776d700 1 Processor -- start 2024-04-26T21:16:39.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb13776d700 1 -- start start 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb13776d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb130101ae0 0x7fb13019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb13776d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1301024a0 0x7fb13019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb13776d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb130107a90 0x7fb130199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb13776d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301124f0 con 0x7fb130107a90 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb13776d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb130112370 con 0x7fb1301024a0 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb13776d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb130112670 con 0x7fb130101ae0 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135d0a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb130107a90 0x7fb130199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135d0a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb130107a90 0x7fb130199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49350/0 (socket says 172.21.15.129:49350) 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135d0a700 1 -- 172.21.15.129:0/161337028 learned_addr learned my addr 172.21.15.129:0/161337028 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb134d08700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1301024a0 0x7fb13019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135509700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb130101ae0 0x7fb13019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135d0a700 1 -- 172.21.15.129:0/161337028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb130101ae0 msgr2=0x7fb13019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135d0a700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb130101ae0 0x7fb13019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135d0a700 1 -- 172.21.15.129:0/161337028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1301024a0 msgr2=0x7fb13019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135d0a700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1301024a0 0x7fb13019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135d0a700 1 -- 172.21.15.129:0/161337028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13019a060 con 0x7fb130107a90 2024-04-26T21:16:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.632+0000 7fb135d0a700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb130107a90 0x7fb130199900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b830 tx=0x7fb128004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.633+0000 7fb1267fc700 1 -- 172.21.15.129:0/161337028 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128021070 con 0x7fb130107a90 2024-04-26T21:16:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.633+0000 7fb1267fc700 1 -- 172.21.15.129:0/161337028 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb128007e30 con 0x7fb130107a90 2024-04-26T21:16:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.633+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13019a2f0 con 0x7fb130107a90 2024-04-26T21:16:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.633+0000 7fb1267fc700 1 -- 172.21.15.129:0/161337028 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128004700 con 0x7fb130107a90 2024-04-26T21:16:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.633+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1301a6610 con 0x7fb130107a90 2024-04-26T21:16:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.633+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb130063b30 con 0x7fb130107a90 2024-04-26T21:16:39.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.635+0000 7fb1267fc700 1 -- 172.21.15.129:0/161337028 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb128003d70 con 0x7fb130107a90 2024-04-26T21:16:39.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.635+0000 7fb1267fc700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb11c0a3d50 0x7fb11c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.635+0000 7fb135509700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb11c0a3d50 0x7fb11c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.635+0000 7fb1267fc700 1 -- 172.21.15.129:0/161337028 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb12800f020 con 0x7fb130107a90 2024-04-26T21:16:39.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.635+0000 7fb1267fc700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] conn(0x7fb11c0ad530 0x7fb11c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:39.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.635+0000 7fb1267fc700 1 -- 172.21.15.129:0/161337028 --> [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb11c0b0000 con 0x7fb11c0ad530 2024-04-26T21:16:39.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.635+0000 7fb1267fc700 1 -- 172.21.15.129:0/161337028 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fb1280d1ec0 con 0x7fb130107a90 2024-04-26T21:16:39.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.636+0000 7fb135509700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb11c0a3d50 0x7fb11c0a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb13019aa40 tx=0x7fb12c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:39.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.636+0000 7fb134d08700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] conn(0x7fb11c0ad530 0x7fb11c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:39.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.636+0000 7fb134d08700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] conn(0x7fb11c0ad530 0x7fb11c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:39.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.637+0000 7fb1267fc700 1 -- 172.21.15.129:0/161337028 <== osd.7 v2:172.21.15.150:6816/1143400463 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb11c0b0000 con 0x7fb11c0ad530 2024-04-26T21:16:39.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.653+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 --> [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb130063f30 con 0x7fb11c0ad530 2024-04-26T21:16:39.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb1267fc700 1 -- 172.21.15.129:0/161337028 <== osd.7 v2:172.21.15.150:6816/1143400463 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb130063f30 con 0x7fb11c0ad530 2024-04-26T21:16:39.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 >> [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] conn(0x7fb11c0ad530 msgr2=0x7fb11c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] conn(0x7fb11c0ad530 0x7fb11c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb11c0a3d50 msgr2=0x7fb11c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb11c0a3d50 0x7fb11c0a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb13019aa40 tx=0x7fb12c0073d0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb130107a90 msgr2=0x7fb130199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 --2- 172.21.15.129:0/161337028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb130107a90 0x7fb130199900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b830 tx=0x7fb128004ce0 comp rx=0 tx=0).stop 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb135d0a700 1 -- 172.21.15.129:0/161337028 reap_dead start 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 shutdown_connections 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 >> 172.21.15.129:0/161337028 conn(0x7fb130078990 msgr2=0x7fb13010f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 shutdown_connections 2024-04-26T21:16:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:39.654+0000 7fb13776d700 1 -- 172.21.15.129:0/161337028 wait complete. 2024-04-26T21:16:40.492 INFO:teuthology.orchestra.run.smithi129.stdout:146028888120 2024-04-26T21:16:40.492 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T21:16:40.596 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:40 smithi129 ceph-mon[29237]: pgmap v308: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:16:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:40 smithi150 ceph-mon[32290]: pgmap v308: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:16:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:40 smithi171 ceph-mon[33478]: pgmap v308: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.917+0000 7f626e334700 1 -- 172.21.15.129:0/2763457326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6268102320 msgr2=0x7f62681027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.917+0000 7f626e334700 1 --2- 172.21.15.129:0/2763457326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6268102320 0x7f62681027a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f625c009990 tx=0x7f625c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.917+0000 7f626e334700 1 -- 172.21.15.129:0/2763457326 shutdown_connections 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.917+0000 7f626e334700 1 --2- 172.21.15.129:0/2763457326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6268102ce0 0x7f626810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.917+0000 7f626e334700 1 --2- 172.21.15.129:0/2763457326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6268108390 0x7f6268108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.917+0000 7f626e334700 1 --2- 172.21.15.129:0/2763457326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6268102320 0x7f62681027a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.917+0000 7f626e334700 1 -- 172.21.15.129:0/2763457326 >> 172.21.15.129:0/2763457326 conn(0x7f62680792d0 msgr2=0x7f62680796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.917+0000 7f626e334700 1 -- 172.21.15.129:0/2763457326 shutdown_connections 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.917+0000 7f626e334700 1 -- 172.21.15.129:0/2763457326 wait complete. 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626e334700 1 Processor -- start 2024-04-26T21:16:40.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626e334700 1 -- start start 2024-04-26T21:16:40.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626e334700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6268102320 0x7f6268198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626e334700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6268102ce0 0x7f6268198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626e334700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6268108390 0x7f6268199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626e334700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6268112530 con 0x7f6268108390 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626e334700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f62681123b0 con 0x7f6268102320 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626e334700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f62681126b0 con 0x7f6268102ce0 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626c8d1700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6268108390 0x7f6268199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626c8d1700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6268108390 0x7f6268199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49374/0 (socket says 172.21.15.129:49374) 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f626c8d1700 1 -- 172.21.15.129:0/3448816790 learned_addr learned my addr 172.21.15.129:0/3448816790 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f62677fe700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6268102ce0 0x7f6268198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.918+0000 7f6267fff700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6268102320 0x7f6268198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f626c8d1700 1 -- 172.21.15.129:0/3448816790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6268102ce0 msgr2=0x7f6268198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f626c8d1700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6268102ce0 0x7f6268198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f626c8d1700 1 -- 172.21.15.129:0/3448816790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6268102320 msgr2=0x7f6268198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f626c8d1700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6268102320 0x7f6268198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f626c8d1700 1 -- 172.21.15.129:0/3448816790 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62681a7ff0 con 0x7f6268108390 2024-04-26T21:16:40.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f6267fff700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6268102320 0x7f6268198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f62677fe700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6268102ce0 0x7f6268198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f626c8d1700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6268108390 0x7f6268199200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6260007ee0 tx=0x7f626000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f62657fa700 1 -- 172.21.15.129:0/3448816790 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626000dbd0 con 0x7f6268108390 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62681a82e0 con 0x7f6268108390 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f62657fa700 1 -- 172.21.15.129:0/3448816790 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f626000dd70 con 0x7f6268108390 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f62657fa700 1 -- 172.21.15.129:0/3448816790 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260005990 con 0x7f6268108390 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.919+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62681a8820 con 0x7f6268108390 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.920+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6268063bb0 con 0x7f6268108390 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.921+0000 7f62657fa700 1 -- 172.21.15.129:0/3448816790 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f626001a020 con 0x7f6268108390 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.921+0000 7f62657fa700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62500a3cd0 0x7f62500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.922+0000 7f6267fff700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62500a3cd0 0x7f62500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.922+0000 7f62657fa700 1 -- 172.21.15.129:0/3448816790 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f62600d0310 con 0x7f6268108390 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.922+0000 7f62657fa700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] conn(0x7f62500ad4b0 0x7f62500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.922+0000 7f62657fa700 1 -- 172.21.15.129:0/3448816790 --> [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6260021e70 con 0x7f62500ad4b0 2024-04-26T21:16:40.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.922+0000 7f62657fa700 1 -- 172.21.15.129:0/3448816790 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f62600d07f0 con 0x7f6268108390 2024-04-26T21:16:40.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.922+0000 7f6267fff700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62500a3cd0 0x7f62500a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6258007900 tx=0x7f6258008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:40.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.922+0000 7f62677fe700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] conn(0x7f62500ad4b0 0x7f62500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.922+0000 7f62677fe700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] conn(0x7f62500ad4b0 0x7f62500af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:40.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.923+0000 7f62657fa700 1 -- 172.21.15.129:0/3448816790 <== osd.1 v2:172.21.15.150:6800/735494308 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6260021e70 con 0x7f62500ad4b0 2024-04-26T21:16:40.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.940+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 --> [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6268064030 con 0x7f62500ad4b0 2024-04-26T21:16:40.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.940+0000 7f62657fa700 1 -- 172.21.15.129:0/3448816790 <== osd.1 v2:172.21.15.150:6800/735494308 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6268064030 con 0x7f62500ad4b0 2024-04-26T21:16:40.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 >> [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] conn(0x7f62500ad4b0 msgr2=0x7f62500af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] conn(0x7f62500ad4b0 0x7f62500af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62500a3cd0 msgr2=0x7f62500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f62500a3cd0 0x7f62500a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6258007900 tx=0x7f6258008040 comp rx=0 tx=0).stop 2024-04-26T21:16:40.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6268108390 msgr2=0x7f6268199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 --2- 172.21.15.129:0/3448816790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6268108390 0x7f6268199200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6260007ee0 tx=0x7f626000d3b0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626c8d1700 1 -- 172.21.15.129:0/3448816790 reap_dead start 2024-04-26T21:16:40.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 shutdown_connections 2024-04-26T21:16:40.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 >> 172.21.15.129:0/3448816790 conn(0x7f62680792d0 msgr2=0x7f6268100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:40.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 shutdown_connections 2024-04-26T21:16:40.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:40.941+0000 7f626e334700 1 -- 172.21.15.129:0/3448816790 wait complete. 2024-04-26T21:16:41.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.773+0000 7f35a4e71700 1 -- 172.21.15.129:0/3147602358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35a0102330 msgr2=0x7f35a01102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:41.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.773+0000 7f35a4e71700 1 --2- 172.21.15.129:0/3147602358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35a0102330 0x7f35a01102f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3590009990 tx=0x7f3590009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:41.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.773+0000 7f35a4e71700 1 -- 172.21.15.129:0/3147602358 shutdown_connections 2024-04-26T21:16:41.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.773+0000 7f35a4e71700 1 --2- 172.21.15.129:0/3147602358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35a0102330 0x7f35a01102f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:41.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.773+0000 7f35a4e71700 1 --2- 172.21.15.129:0/3147602358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a01094b0 0x7f35a0109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:41.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.773+0000 7f35a4e71700 1 --2- 172.21.15.129:0/3147602358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a0101970 0x7f35a0101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:41.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.773+0000 7f35a4e71700 1 -- 172.21.15.129:0/3147602358 >> 172.21.15.129:0/3147602358 conn(0x7f35a0078920 msgr2=0x7f35a00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:41.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.773+0000 7f35a4e71700 1 -- 172.21.15.129:0/3147602358 shutdown_connections 2024-04-26T21:16:41.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.774+0000 7f35a4e71700 1 -- 172.21.15.129:0/3147602358 wait complete. 2024-04-26T21:16:41.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.774+0000 7f35a4e71700 1 Processor -- start 2024-04-26T21:16:41.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35a4e71700 1 -- start start 2024-04-26T21:16:41.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35a4e71700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35a0101970 0x7f35a019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35a4e71700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a0102330 0x7f35a019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35a4e71700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a01094b0 0x7f35a01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35a4e71700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01135f0 con 0x7f35a0101970 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35a4e71700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0113470 con 0x7f35a01094b0 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35a4e71700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0113770 con 0x7f35a0102330 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359e59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35a0101970 0x7f35a019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359e59c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35a0101970 0x7f35a019f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49392/0 (socket says 172.21.15.129:49392) 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359e59c700 1 -- 172.21.15.129:0/2844995766 learned_addr learned my addr 172.21.15.129:0/2844995766 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359ed9d700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a01094b0 0x7f35a01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359dd9b700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a0102330 0x7f35a019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359e59c700 1 -- 172.21.15.129:0/2844995766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a0102330 msgr2=0x7f35a019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359e59c700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a0102330 0x7f35a019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:41.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359e59c700 1 -- 172.21.15.129:0/2844995766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a01094b0 msgr2=0x7f35a01998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359e59c700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a01094b0 0x7f35a01998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359e59c700 1 -- 172.21.15.129:0/2844995766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a019a050 con 0x7f35a0101970 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359dd9b700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a0102330 0x7f35a019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f359e59c700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35a0101970 0x7f35a019f6e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f359800ef10 tx=0x7f359800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35977fe700 1 -- 172.21.15.129:0/2844995766 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359800ca70 con 0x7f35a0101970 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a019a2e0 con 0x7f35a0101970 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35977fe700 1 -- 172.21.15.129:0/2844995766 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3598019070 con 0x7f35a0101970 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.775+0000 7f35977fe700 1 -- 172.21.15.129:0/2844995766 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598014480 con 0x7f35a0101970 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.776+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a01a65b0 con 0x7f35a0101970 2024-04-26T21:16:41.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.776+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35a0063b70 con 0x7f35a0101970 2024-04-26T21:16:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.778+0000 7f35977fe700 1 -- 172.21.15.129:0/2844995766 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3598012050 con 0x7f35a0101970 2024-04-26T21:16:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.778+0000 7f35977fe700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f358c0a3cd0 0x7f358c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.778+0000 7f359dd9b700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f358c0a3cd0 0x7f358c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.778+0000 7f35977fe700 1 -- 172.21.15.129:0/2844995766 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f35980d0ec0 con 0x7f35a0101970 2024-04-26T21:16:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.778+0000 7f35977fe700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] conn(0x7f358c0ad4b0 0x7f358c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.778+0000 7f35977fe700 1 -- 172.21.15.129:0/2844995766 --> [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3598008ec0 con 0x7f358c0ad4b0 2024-04-26T21:16:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.779+0000 7f35977fe700 1 -- 172.21.15.129:0/2844995766 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f35980d13a0 con 0x7f35a0101970 2024-04-26T21:16:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.779+0000 7f359ed9d700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] conn(0x7f358c0ad4b0 0x7f358c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.779+0000 7f359dd9b700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f358c0a3cd0 0x7f358c0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f35a0108bd0 tx=0x7f358800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.779+0000 7f359ed9d700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] conn(0x7f358c0ad4b0 0x7f358c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.780+0000 7f35977fe700 1 -- 172.21.15.129:0/2844995766 <== osd.0 v2:172.21.15.129:6802/3232878544 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3598008ec0 con 0x7f358c0ad4b0 2024-04-26T21:16:41.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.796+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 --> [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35a0063f70 con 0x7f358c0ad4b0 2024-04-26T21:16:41.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.796+0000 7f35977fe700 1 -- 172.21.15.129:0/2844995766 <== osd.0 v2:172.21.15.129:6802/3232878544 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f35a0063f70 con 0x7f358c0ad4b0 2024-04-26T21:16:41.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] conn(0x7f358c0ad4b0 msgr2=0x7f358c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:41.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] conn(0x7f358c0ad4b0 0x7f358c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:41.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f358c0a3cd0 msgr2=0x7f358c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:41.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f358c0a3cd0 0x7f358c0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f35a0108bd0 tx=0x7f358800b2c0 comp rx=0 tx=0).stop 2024-04-26T21:16:41.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35a0101970 msgr2=0x7f35a019f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:41.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 --2- 172.21.15.129:0/2844995766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35a0101970 0x7f35a019f6e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f359800ef10 tx=0x7f359800c3a0 comp rx=0 tx=0).stop 2024-04-26T21:16:41.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f359ed9d700 1 -- 172.21.15.129:0/2844995766 reap_dead start 2024-04-26T21:16:41.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 shutdown_connections 2024-04-26T21:16:41.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 >> 172.21.15.129:0/2844995766 conn(0x7f35a0078920 msgr2=0x7f35a01000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:41.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 shutdown_connections 2024-04-26T21:16:41.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:41.797+0000 7f35a4e71700 1 -- 172.21.15.129:0/2844995766 wait complete. 2024-04-26T21:16:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:42 smithi150 ceph-mon[32290]: pgmap v309: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T21:16:42.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:42 smithi129 ceph-mon[29237]: pgmap v309: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T21:16:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:42 smithi171 ceph-mon[33478]: pgmap v309: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T21:16:42.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.030+0000 7fe924e90700 1 -- 172.21.15.129:0/380109517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe920108380 msgr2=0x7fe920108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:42.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.030+0000 7fe924e90700 1 --2- 172.21.15.129:0/380109517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe920108380 0x7fe920108760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe910009990 tx=0x7fe910009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:42.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.030+0000 7fe924e90700 1 -- 172.21.15.129:0/380109517 shutdown_connections 2024-04-26T21:16:42.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.030+0000 7fe924e90700 1 --2- 172.21.15.129:0/380109517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe920102cd0 0x7fe92010f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:42.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.030+0000 7fe924e90700 1 --2- 172.21.15.129:0/380109517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe920108380 0x7fe920108760 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.030+0000 7fe924e90700 1 --2- 172.21.15.129:0/380109517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe920102310 0x7fe920102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.030+0000 7fe924e90700 1 -- 172.21.15.129:0/380109517 >> 172.21.15.129:0/380109517 conn(0x7fe920079290 msgr2=0x7fe9200796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.030+0000 7fe924e90700 1 -- 172.21.15.129:0/380109517 shutdown_connections 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.030+0000 7fe924e90700 1 -- 172.21.15.129:0/380109517 wait complete. 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.031+0000 7fe924e90700 1 Processor -- start 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.031+0000 7fe924e90700 1 -- start start 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.031+0000 7fe924e90700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe920102310 0x7fe92019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.031+0000 7fe924e90700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe920102cd0 0x7fe92019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.031+0000 7fe924e90700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe920108380 0x7fe9201a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.031+0000 7fe924e90700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe920112520 con 0x7fe920102cd0 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.031+0000 7fe924e90700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201123a0 con 0x7fe920108380 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.031+0000 7fe924e90700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201126a0 con 0x7fe920102310 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.031+0000 7fe91dd9b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe920102cd0 0x7fe92019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:42.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91dd9b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe920102cd0 0x7fe92019af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49420/0 (socket says 172.21.15.129:49420) 2024-04-26T21:16:42.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91dd9b700 1 -- 172.21.15.129:0/1038757063 learned_addr learned my addr 172.21.15.129:0/1038757063 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:42.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91e59c700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe920102310 0x7fe92019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:42.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91ed9d700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe920108380 0x7fe9201a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:42.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91dd9b700 1 -- 172.21.15.129:0/1038757063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe920102310 msgr2=0x7fe92019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:42.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91dd9b700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe920102310 0x7fe92019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:42.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91dd9b700 1 -- 172.21.15.129:0/1038757063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe920108380 msgr2=0x7fe9201a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91dd9b700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe920108380 0x7fe9201a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91dd9b700 1 -- 172.21.15.129:0/1038757063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9201a5fe0 con 0x7fe920102cd0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe91dd9b700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe920102cd0 0x7fe92019af60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe91400d950 tx=0x7fe91400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe90f7fe700 1 -- 172.21.15.129:0/1038757063 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914016070 con 0x7fe920102cd0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9201a6210 con 0x7fe920102cd0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe90f7fe700 1 -- 172.21.15.129:0/1038757063 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe91400fd40 con 0x7fe920102cd0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe90f7fe700 1 -- 172.21.15.129:0/1038757063 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91400e3b0 con 0x7fe920102cd0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.032+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9201a6750 con 0x7fe920102cd0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.033+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe920063b30 con 0x7fe920102cd0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.034+0000 7fe90f7fe700 1 -- 172.21.15.129:0/1038757063 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe914019070 con 0x7fe920102cd0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.035+0000 7fe90f7fe700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe9080a3cd0 0x7fe9080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.035+0000 7fe91e59c700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe9080a3cd0 0x7fe9080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.035+0000 7fe90f7fe700 1 -- 172.21.15.129:0/1038757063 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe9140d03c0 con 0x7fe920102cd0 2024-04-26T21:16:42.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.035+0000 7fe90f7fe700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] conn(0x7fe9080ad4b0 0x7fe9080af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.035+0000 7fe90f7fe700 1 -- 172.21.15.129:0/1038757063 --> [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe914013ec0 con 0x7fe9080ad4b0 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.035+0000 7fe90f7fe700 1 -- 172.21.15.129:0/1038757063 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe9140d08a0 con 0x7fe920102cd0 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.035+0000 7fe91e59c700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe9080a3cd0 0x7fe9080a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe91000b5c0 tx=0x7fe910005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.035+0000 7fe91ed9d700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] conn(0x7fe9080ad4b0 0x7fe9080af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.036+0000 7fe91ed9d700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] conn(0x7fe9080ad4b0 0x7fe9080af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.037+0000 7fe90f7fe700 1 -- 172.21.15.129:0/1038757063 <== osd.10 v2:172.21.15.150:6824/1187635399 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe914013ec0 con 0x7fe9080ad4b0 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.053+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 --> [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe920063f30 con 0x7fe9080ad4b0 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.053+0000 7fe90f7fe700 1 -- 172.21.15.129:0/1038757063 <== osd.10 v2:172.21.15.150:6824/1187635399 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe920063f30 con 0x7fe9080ad4b0 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 >> [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] conn(0x7fe9080ad4b0 msgr2=0x7fe9080af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] conn(0x7fe9080ad4b0 0x7fe9080af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe9080a3cd0 msgr2=0x7fe9080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe9080a3cd0 0x7fe9080a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe91000b5c0 tx=0x7fe910005960 comp rx=0 tx=0).stop 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe920102cd0 msgr2=0x7fe92019af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:42.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 --2- 172.21.15.129:0/1038757063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe920102cd0 0x7fe92019af60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe91400d950 tx=0x7fe91400dc60 comp rx=0 tx=0).stop 2024-04-26T21:16:42.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe91ed9d700 1 -- 172.21.15.129:0/1038757063 reap_dead start 2024-04-26T21:16:42.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 shutdown_connections 2024-04-26T21:16:42.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 >> 172.21.15.129:0/1038757063 conn(0x7fe920079290 msgr2=0x7fe920101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:42.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 shutdown_connections 2024-04-26T21:16:42.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:42.054+0000 7fe924e90700 1 -- 172.21.15.129:0/1038757063 wait complete. 2024-04-26T21:16:43.938 INFO:teuthology.orchestra.run.smithi129.stdout:223338299433 2024-04-26T21:16:43.938 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T21:16:44.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:44 smithi150 ceph-mon[32290]: pgmap v310: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:16:44.649 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:44 smithi129 ceph-mon[29237]: pgmap v310: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:16:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:44 smithi171 ceph-mon[33478]: pgmap v310: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:16:45.226 INFO:teuthology.orchestra.run.smithi129.stdout:55834574920 2024-04-26T21:16:45.226 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T21:16:45.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:45 smithi129 ceph-mon[29237]: pgmap v311: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:45.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:45 smithi171 ceph-mon[33478]: pgmap v311: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:45 smithi150 ceph-mon[32290]: pgmap v311: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:46.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.868+0000 7f9709815700 1 -- 172.21.15.129:0/3339832769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102f60 msgr2=0x7f9704106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:46.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.868+0000 7f9709815700 1 --2- 172.21.15.129:0/3339832769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102f60 0x7f9704106f60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f96f80099c0 tx=0x7f96f8009cd0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.868+0000 7f9709815700 1 -- 172.21.15.129:0/3339832769 shutdown_connections 2024-04-26T21:16:46.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.868+0000 7f9709815700 1 --2- 172.21.15.129:0/3339832769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9704107640 0x7f970410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.868+0000 7f9709815700 1 --2- 172.21.15.129:0/3339832769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97041025b0 0x7f9704102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.868+0000 7f9709815700 1 --2- 172.21.15.129:0/3339832769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102f60 0x7f9704106f60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.868+0000 7f9709815700 1 -- 172.21.15.129:0/3339832769 >> 172.21.15.129:0/3339832769 conn(0x7f97040792d0 msgr2=0x7f97040796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:46.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.868+0000 7f9709815700 1 -- 172.21.15.129:0/3339832769 shutdown_connections 2024-04-26T21:16:46.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.868+0000 7f9709815700 1 -- 172.21.15.129:0/3339832769 wait complete. 2024-04-26T21:16:46.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.869+0000 7f9709815700 1 Processor -- start 2024-04-26T21:16:46.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.869+0000 7f9709815700 1 -- start start 2024-04-26T21:16:46.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9709815700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97041025b0 0x7f970419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:46.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9709815700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9704102f60 0x7f970419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:46.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9709815700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9704107640 0x7f97041989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:46.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9709815700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9704112600 con 0x7f97041025b0 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9709815700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9704112480 con 0x7f9704102f60 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9709815700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9704112780 con 0x7f9704107640 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9702ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97041025b0 0x7f970419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9702ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97041025b0 0x7f970419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39702/0 (socket says 172.21.15.129:39702) 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9702ffd700 1 -- 172.21.15.129:0/2070481436 learned_addr learned my addr 172.21.15.129:0/2070481436 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f97027fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9704102f60 0x7f970419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f97037fe700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9704107640 0x7f97041989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9702ffd700 1 -- 172.21.15.129:0/2070481436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9704107640 msgr2=0x7f97041989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9702ffd700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9704107640 0x7f97041989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9702ffd700 1 -- 172.21.15.129:0/2070481436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9704102f60 msgr2=0x7f970419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9702ffd700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9704102f60 0x7f970419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9702ffd700 1 -- 172.21.15.129:0/2070481436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9704199120 con 0x7f97041025b0 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f97027fc700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9704102f60 0x7f970419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.870+0000 7f9702ffd700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97041025b0 0x7f970419e7b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f96f400d9e0 tx=0x7f96f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:46.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.871+0000 7f9708813700 1 -- 172.21.15.129:0/2070481436 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f401b070 con 0x7f97041025b0 2024-04-26T21:16:46.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.871+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9704199410 con 0x7f97041025b0 2024-04-26T21:16:46.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.871+0000 7f9708813700 1 -- 172.21.15.129:0/2070481436 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f400eb70 con 0x7f97041025b0 2024-04-26T21:16:46.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.871+0000 7f9708813700 1 -- 172.21.15.129:0/2070481436 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f40145a0 con 0x7f97041025b0 2024-04-26T21:16:46.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.871+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970406bb50 con 0x7f97041025b0 2024-04-26T21:16:46.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.872+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f970406c160 con 0x7f97041025b0 2024-04-26T21:16:46.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.873+0000 7f9708813700 1 -- 172.21.15.129:0/2070481436 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f96f401e070 con 0x7f97041025b0 2024-04-26T21:16:46.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.874+0000 7f9708813700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96ec0a3cd0 0x7f96ec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:46.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.874+0000 7f97027fc700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96ec0a3cd0 0x7f96ec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:46.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.874+0000 7f9708813700 1 -- 172.21.15.129:0/2070481436 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f96f40d1910 con 0x7f97041025b0 2024-04-26T21:16:46.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.874+0000 7f9708813700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] conn(0x7f96ec0ad4b0 0x7f96ec0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:46.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.874+0000 7f9708813700 1 -- 172.21.15.129:0/2070481436 --> [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96ec0aff80 con 0x7f96ec0ad4b0 2024-04-26T21:16:46.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.874+0000 7f9708813700 1 -- 172.21.15.129:0/2070481436 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f96f40d62a0 con 0x7f97041025b0 2024-04-26T21:16:46.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.874+0000 7f97027fc700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96ec0a3cd0 0x7f96ec0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f96f8006010 tx=0x7f96f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:46.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.874+0000 7f97037fe700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] conn(0x7f96ec0ad4b0 0x7f96ec0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:46.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.875+0000 7f97037fe700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] conn(0x7f96ec0ad4b0 0x7f96ec0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:46.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.876+0000 7f9708813700 1 -- 172.21.15.129:0/2070481436 <== osd.8 v2:172.21.15.171:6816/2306529186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96ec0aff80 con 0x7f96ec0ad4b0 2024-04-26T21:16:46.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.894+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 --> [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f970406c5e0 con 0x7f96ec0ad4b0 2024-04-26T21:16:46.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.895+0000 7f9708813700 1 -- 172.21.15.129:0/2070481436 <== osd.8 v2:172.21.15.171:6816/2306529186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f970406c5e0 con 0x7f96ec0ad4b0 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.895+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 >> [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] conn(0x7f96ec0ad4b0 msgr2=0x7f96ec0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.895+0000 7f9709815700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] conn(0x7f96ec0ad4b0 0x7f96ec0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.895+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96ec0a3cd0 msgr2=0x7f96ec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.895+0000 7f9709815700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96ec0a3cd0 0x7f96ec0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f96f8006010 tx=0x7f96f80058e0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.895+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97041025b0 msgr2=0x7f970419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.895+0000 7f9709815700 1 --2- 172.21.15.129:0/2070481436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97041025b0 0x7f970419e7b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f96f400d9e0 tx=0x7f96f400dcf0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.895+0000 7f97037fe700 1 -- 172.21.15.129:0/2070481436 reap_dead start 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.896+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 shutdown_connections 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.896+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 >> 172.21.15.129:0/2070481436 conn(0x7f97040792d0 msgr2=0x7f97041004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.896+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 shutdown_connections 2024-04-26T21:16:46.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:46.896+0000 7f9709815700 1 -- 172.21.15.129:0/2070481436 wait complete. 2024-04-26T21:16:47.332 INFO:teuthology.orchestra.run.smithi129.stdout:313532612634 2024-04-26T21:16:47.333 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T21:16:48.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 -- 172.21.15.129:0/3671167615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5dc108390 msgr2=0x7fb5dc108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 --2- 172.21.15.129:0/3671167615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5dc108390 0x7fb5dc108770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009990 tx=0x7fb5cc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 -- 172.21.15.129:0/3671167615 shutdown_connections 2024-04-26T21:16:48.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 --2- 172.21.15.129:0/3671167615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5dc102ce0 0x7fb5dc10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 --2- 172.21.15.129:0/3671167615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5dc108390 0x7fb5dc108770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 --2- 172.21.15.129:0/3671167615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc102320 0x7fb5dc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 -- 172.21.15.129:0/3671167615 >> 172.21.15.129:0/3671167615 conn(0x7fb5dc0792d0 msgr2=0x7fb5dc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 -- 172.21.15.129:0/3671167615 shutdown_connections 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 -- 172.21.15.129:0/3671167615 wait complete. 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.436+0000 7fb5e1a73700 1 Processor -- start 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5e1a73700 1 -- start start 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5e1a73700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5dc102320 0x7fb5dc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5e1a73700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5dc102ce0 0x7fb5dc19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5e1a73700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc108390 0x7fb5dc1a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5e1a73700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc112560 con 0x7fb5dc102ce0 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5e1a73700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc1123e0 con 0x7fb5dc108390 2024-04-26T21:16:48.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5e1a73700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc1126e0 con 0x7fb5dc102320 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5daffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5dc102ce0 0x7fb5dc19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5daffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5dc102ce0 0x7fb5dc19afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39726/0 (socket says 172.21.15.129:39726) 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5daffd700 1 -- 172.21.15.129:0/541999434 learned_addr learned my addr 172.21.15.129:0/541999434 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5dbfff700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc108390 0x7fb5dc1a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5db7fe700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5dc102320 0x7fb5dc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5daffd700 1 -- 172.21.15.129:0/541999434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5dc102320 msgr2=0x7fb5dc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5daffd700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5dc102320 0x7fb5dc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5daffd700 1 -- 172.21.15.129:0/541999434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc108390 msgr2=0x7fb5dc1a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5daffd700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc108390 0x7fb5dc1a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5daffd700 1 -- 172.21.15.129:0/541999434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc1a6040 con 0x7fb5dc102ce0 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5db7fe700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5dc102320 0x7fb5dc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.437+0000 7fb5daffd700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5dc102ce0 0x7fb5dc19afc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000c970 tx=0x7fb5d000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.438+0000 7fb5d8ff9700 1 -- 172.21.15.129:0/541999434 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0007aa0 con 0x7fb5dc102ce0 2024-04-26T21:16:48.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.438+0000 7fb5d8ff9700 1 -- 172.21.15.129:0/541999434 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d0007c40 con 0x7fb5dc102ce0 2024-04-26T21:16:48.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.438+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc1a6270 con 0x7fb5dc102ce0 2024-04-26T21:16:48.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.438+0000 7fb5d8ff9700 1 -- 172.21.15.129:0/541999434 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d000f6b0 con 0x7fb5dc102ce0 2024-04-26T21:16:48.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.438+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc1a6780 con 0x7fb5dc102ce0 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.439+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5c8001030 con 0x7fb5dc102ce0 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.440+0000 7fb5d8ff9700 1 -- 172.21.15.129:0/541999434 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb5d00213f0 con 0x7fb5dc102ce0 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.441+0000 7fb5d8ff9700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb5c40a3d50 0x7fb5c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.441+0000 7fb5db7fe700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb5c40a3d50 0x7fb5c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.441+0000 7fb5d8ff9700 1 -- 172.21.15.129:0/541999434 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb5d00d0bc0 con 0x7fb5dc102ce0 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.441+0000 7fb5d8ff9700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] conn(0x7fb5c40ad530 0x7fb5c40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.441+0000 7fb5d8ff9700 1 -- 172.21.15.129:0/541999434 --> [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5d0022e50 con 0x7fb5c40ad530 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.441+0000 7fb5d8ff9700 1 -- 172.21.15.129:0/541999434 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fb5d00d10a0 con 0x7fb5dc102ce0 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.442+0000 7fb5dbfff700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] conn(0x7fb5c40ad530 0x7fb5c40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.442+0000 7fb5db7fe700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb5c40a3d50 0x7fb5c40a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006010 tx=0x7fb5cc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:48.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.442+0000 7fb5dbfff700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] conn(0x7fb5c40ad530 0x7fb5c40af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:48.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.444+0000 7fb5d8ff9700 1 -- 172.21.15.129:0/541999434 <== osd.9 v2:172.21.15.129:6826/3561447119 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5d0022e50 con 0x7fb5c40ad530 2024-04-26T21:16:48.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.460+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 --> [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb5c8002e60 con 0x7fb5c40ad530 2024-04-26T21:16:48.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5d8ff9700 1 -- 172.21.15.129:0/541999434 <== osd.9 v2:172.21.15.129:6826/3561447119 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb5c8002e60 con 0x7fb5c40ad530 2024-04-26T21:16:48.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] conn(0x7fb5c40ad530 msgr2=0x7fb5c40af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] conn(0x7fb5c40ad530 0x7fb5c40af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb5c40a3d50 msgr2=0x7fb5c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb5c40a3d50 0x7fb5c40a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006010 tx=0x7fb5cc00b540 comp rx=0 tx=0).stop 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5dc102ce0 msgr2=0x7fb5dc19afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 --2- 172.21.15.129:0/541999434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5dc102ce0 0x7fb5dc19afc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000c970 tx=0x7fb5d000cc80 comp rx=0 tx=0).stop 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5dbfff700 1 -- 172.21.15.129:0/541999434 reap_dead start 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 shutdown_connections 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 >> 172.21.15.129:0/541999434 conn(0x7fb5dc0792d0 msgr2=0x7fb5dc1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 shutdown_connections 2024-04-26T21:16:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.461+0000 7fb5e1a73700 1 -- 172.21.15.129:0/541999434 wait complete. 2024-04-26T21:16:48.619 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:48 smithi129 ceph-mon[29237]: pgmap v312: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:48.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:48 smithi150 ceph-mon[32290]: pgmap v312: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:48.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:48 smithi171 ceph-mon[33478]: pgmap v312: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:48.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.720+0000 7f6e23d74700 1 -- 172.21.15.129:0/2589576954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e1c1094c0 msgr2=0x7f6e1c1098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.720+0000 7f6e23d74700 1 --2- 172.21.15.129:0/2589576954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e1c1094c0 0x7f6e1c1098a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009990 tx=0x7f6e18009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.720+0000 7f6e23d74700 1 -- 172.21.15.129:0/2589576954 shutdown_connections 2024-04-26T21:16:48.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.720+0000 7f6e23d74700 1 --2- 172.21.15.129:0/2589576954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e1c103e30 0x7f6e1c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.720+0000 7f6e23d74700 1 --2- 172.21.15.129:0/2589576954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e1c1094c0 0x7f6e1c1098a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.720+0000 7f6e23d74700 1 --2- 172.21.15.129:0/2589576954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e1c103470 0x7f6e1c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.720+0000 7f6e23d74700 1 -- 172.21.15.129:0/2589576954 >> 172.21.15.129:0/2589576954 conn(0x7f6e1c07a580 msgr2=0x7f6e1c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.720+0000 7f6e23d74700 1 -- 172.21.15.129:0/2589576954 shutdown_connections 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e23d74700 1 -- 172.21.15.129:0/2589576954 wait complete. 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e23d74700 1 Processor -- start 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e23d74700 1 -- start start 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e23d74700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e1c103470 0x7f6e1c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e23d74700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e1c103e30 0x7f6e1c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e23d74700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e1c1094c0 0x7f6e1c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e23d74700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1135f0 con 0x7f6e1c1094c0 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e23d74700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c113470 con 0x7f6e1c103e30 2024-04-26T21:16:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e23d74700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c113770 con 0x7f6e1c103470 2024-04-26T21:16:48.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e22311700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e1c1094c0 0x7f6e1c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e22311700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e1c1094c0 0x7f6e1c199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39746/0 (socket says 172.21.15.129:39746) 2024-04-26T21:16:48.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e22311700 1 -- 172.21.15.129:0/353888332 learned_addr learned my addr 172.21.15.129:0/353888332 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:48.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e21b10700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e1c103470 0x7f6e1c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.721+0000 7f6e2130f700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e1c103e30 0x7f6e1c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e22311700 1 -- 172.21.15.129:0/353888332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e1c103470 msgr2=0x7f6e1c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e22311700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e1c103470 0x7f6e1c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e22311700 1 -- 172.21.15.129:0/353888332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e1c103e30 msgr2=0x7f6e1c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e22311700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e1c103e30 0x7f6e1c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e22311700 1 -- 172.21.15.129:0/353888332 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1c19a0d0 con 0x7f6e1c1094c0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e22311700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e1c1094c0 0x7f6e1c199970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6e14007ee0 tx=0x7f6e1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e12ffd700 1 -- 172.21.15.129:0/353888332 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14004bb0 con 0x7f6e1c1094c0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1c19a3c0 con 0x7f6e1c1094c0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e12ffd700 1 -- 172.21.15.129:0/353888332 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e14004d50 con 0x7f6e1c1094c0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e12ffd700 1 -- 172.21.15.129:0/353888332 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14005920 con 0x7f6e1c1094c0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.722+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1c1a6680 con 0x7f6e1c1094c0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.723+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e1c063b70 con 0x7f6e1c1094c0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.724+0000 7f6e12ffd700 1 -- 172.21.15.129:0/353888332 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6e1401a020 con 0x7f6e1c1094c0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.725+0000 7f6e12ffd700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6e080a3cd0 0x7f6e080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.725+0000 7f6e21b10700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6e080a3cd0 0x7f6e080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.725+0000 7f6e12ffd700 1 -- 172.21.15.129:0/353888332 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6e140d00d0 con 0x7f6e1c1094c0 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.725+0000 7f6e12ffd700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] conn(0x7f6e080ad4b0 0x7f6e080af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.725+0000 7f6e12ffd700 1 -- 172.21.15.129:0/353888332 --> [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e080aff80 con 0x7f6e080ad4b0 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.725+0000 7f6e12ffd700 1 -- 172.21.15.129:0/353888332 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f6e140d05b0 con 0x7f6e1c1094c0 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.725+0000 7f6e2130f700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] conn(0x7f6e080ad4b0 0x7f6e080af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.725+0000 7f6e21b10700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6e080a3cd0 0x7f6e080a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800b5c0 tx=0x7f6e18005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.725+0000 7f6e2130f700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] conn(0x7f6e080ad4b0 0x7f6e080af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.726+0000 7f6e12ffd700 1 -- 172.21.15.129:0/353888332 <== osd.3 v2:172.21.15.129:6810/347897303 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e080aff80 con 0x7f6e080ad4b0 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.742+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 --> [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e1c063ff0 con 0x7f6e080ad4b0 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.742+0000 7f6e12ffd700 1 -- 172.21.15.129:0/353888332 <== osd.3 v2:172.21.15.129:6810/347897303 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6e1c063ff0 con 0x7f6e080ad4b0 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] conn(0x7f6e080ad4b0 msgr2=0x7f6e080af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] conn(0x7f6e080ad4b0 0x7f6e080af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6e080a3cd0 msgr2=0x7f6e080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6e080a3cd0 0x7f6e080a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800b5c0 tx=0x7f6e18005960 comp rx=0 tx=0).stop 2024-04-26T21:16:48.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e1c1094c0 msgr2=0x7f6e1c199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:48.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 --2- 172.21.15.129:0/353888332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e1c1094c0 0x7f6e1c199970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6e14007ee0 tx=0x7f6e1400d3b0 comp rx=0 tx=0).stop 2024-04-26T21:16:48.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e22311700 1 -- 172.21.15.129:0/353888332 reap_dead start 2024-04-26T21:16:48.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 shutdown_connections 2024-04-26T21:16:48.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 >> 172.21.15.129:0/353888332 conn(0x7f6e1c07a580 msgr2=0x7f6e1c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:48.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 shutdown_connections 2024-04-26T21:16:48.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:48.743+0000 7f6e23d74700 1 -- 172.21.15.129:0/353888332 wait complete. 2024-04-26T21:16:49.974 INFO:teuthology.orchestra.run.smithi129.stdout:292057776161 2024-04-26T21:16:49.974 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T21:16:50.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 -- 172.21.15.129:0/2063478840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbea41094a0 msgr2=0x7fbea4109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:50.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 --2- 172.21.15.129:0/2063478840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbea41094a0 0x7fbea4109880 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009990 tx=0x7fbe94009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:50.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 -- 172.21.15.129:0/2063478840 shutdown_connections 2024-04-26T21:16:50.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 --2- 172.21.15.129:0/2063478840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c510 0x7fbea41102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 --2- 172.21.15.129:0/2063478840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbea41094a0 0x7fbea4109880 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 --2- 172.21.15.129:0/2063478840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbea406bb70 0x7fbea406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 -- 172.21.15.129:0/2063478840 >> 172.21.15.129:0/2063478840 conn(0x7fbea40792f0 msgr2=0x7fbea4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 -- 172.21.15.129:0/2063478840 shutdown_connections 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 -- 172.21.15.129:0/2063478840 wait complete. 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 Processor -- start 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.622+0000 7fbea9979700 1 -- start start 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea9979700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbea406bb70 0x7fbea4102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea9979700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c510 0x7fbea4102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea9979700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbea41094a0 0x7fbea4102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea9979700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbea41135b0 con 0x7fbea41094a0 2024-04-26T21:16:50.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea9979700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4113430 con 0x7fbea406c510 2024-04-26T21:16:50.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea9979700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4113730 con 0x7fbea406bb70 2024-04-26T21:16:50.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea37fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbea41094a0 0x7fbea4102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:50.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea37fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbea41094a0 0x7fbea4102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39758/0 (socket says 172.21.15.129:39758) 2024-04-26T21:16:50.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea37fe700 1 -- 172.21.15.129:0/3107335471 learned_addr learned my addr 172.21.15.129:0/3107335471 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:50.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea2ffd700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbea406bb70 0x7fbea4102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:50.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbe9a7fc700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c510 0x7fbea4102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea37fe700 1 -- 172.21.15.129:0/3107335471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbea406bb70 msgr2=0x7fbea4102420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea37fe700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbea406bb70 0x7fbea4102420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea37fe700 1 -- 172.21.15.129:0/3107335471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c510 msgr2=0x7fbea4102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea37fe700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c510 0x7fbea4102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea37fe700 1 -- 172.21.15.129:0/3107335471 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea41a7ff0 con 0x7fbea41094a0 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea37fe700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbea41094a0 0x7fbea4102ea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007d70 tx=0x7fbe9c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.623+0000 7fbea8977700 1 -- 172.21.15.129:0/3107335471 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c016020 con 0x7fbea41094a0 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.624+0000 7fbea8977700 1 -- 172.21.15.129:0/3107335471 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9c00ad40 con 0x7fbea41094a0 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.624+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea41a82e0 con 0x7fbea41094a0 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.624+0000 7fbea8977700 1 -- 172.21.15.129:0/3107335471 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0115e0 con 0x7fbea41094a0 2024-04-26T21:16:50.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.624+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea41a87d0 con 0x7fbea41094a0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.624+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbea4063b30 con 0x7fbea41094a0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.626+0000 7fbea8977700 1 -- 172.21.15.129:0/3107335471 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbe9c019020 con 0x7fbea41094a0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.626+0000 7fbea8977700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe900a3cd0 0x7fbe900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.626+0000 7fbea2ffd700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe900a3cd0 0x7fbe900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.626+0000 7fbea8977700 1 -- 172.21.15.129:0/3107335471 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbe9c0d0820 con 0x7fbea41094a0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.626+0000 7fbea8977700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] conn(0x7fbe900ad4b0 0x7fbe900af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.626+0000 7fbea8977700 1 -- 172.21.15.129:0/3107335471 --> [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe9c027e50 con 0x7fbe900ad4b0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.626+0000 7fbea8977700 1 -- 172.21.15.129:0/3107335471 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fbe9c0d0d00 con 0x7fbea41094a0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.627+0000 7fbea2ffd700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe900a3cd0 0x7fbe900a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400b5c0 tx=0x7fbe94005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.627+0000 7fbe9a7fc700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] conn(0x7fbe900ad4b0 0x7fbe900af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.627+0000 7fbe9a7fc700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] conn(0x7fbe900ad4b0 0x7fbe900af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.628+0000 7fbea8977700 1 -- 172.21.15.129:0/3107335471 <== osd.5 v2:172.21.15.171:6808/1324309916 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe9c027e50 con 0x7fbe900ad4b0 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.644+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 --> [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbea4063fb0 con 0x7fbe900ad4b0 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea8977700 1 -- 172.21.15.129:0/3107335471 <== osd.5 v2:172.21.15.171:6808/1324309916 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbea4063fb0 con 0x7fbe900ad4b0 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 >> [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] conn(0x7fbe900ad4b0 msgr2=0x7fbe900af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea9979700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] conn(0x7fbe900ad4b0 0x7fbe900af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe900a3cd0 msgr2=0x7fbe900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea9979700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbe900a3cd0 0x7fbe900a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400b5c0 tx=0x7fbe94005960 comp rx=0 tx=0).stop 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbea41094a0 msgr2=0x7fbea4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea9979700 1 --2- 172.21.15.129:0/3107335471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbea41094a0 0x7fbea4102ea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007d70 tx=0x7fbe9c00a340 comp rx=0 tx=0).stop 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea37fe700 1 -- 172.21.15.129:0/3107335471 reap_dead start 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 shutdown_connections 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.645+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 >> 172.21.15.129:0/3107335471 conn(0x7fbea40792f0 msgr2=0x7fbea4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.646+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 shutdown_connections 2024-04-26T21:16:50.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:49.646+0000 7fbea9979700 1 -- 172.21.15.129:0/3107335471 wait complete. 2024-04-26T21:16:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:50 smithi150 ceph-mon[32290]: pgmap v313: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:50.703 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:50 smithi129 ceph-mon[29237]: pgmap v313: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:50.734 INFO:teuthology.orchestra.run.smithi129.stdout:34359738445 2024-04-26T21:16:50.735 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T21:16:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:50 smithi171 ceph-mon[33478]: pgmap v313: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:51.166 INFO:teuthology.orchestra.run.smithi129.stdout:111669149759 2024-04-26T21:16:51.166 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T21:16:51.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.493+0000 7fb4a910e700 1 -- 172.21.15.129:0/1347790306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a4102f50 msgr2=0x7fb4a4106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.493+0000 7fb4a910e700 1 --2- 172.21.15.129:0/1347790306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a4102f50 0x7fb4a4106fa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4980099c0 tx=0x7fb498009cd0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.493+0000 7fb4a910e700 1 -- 172.21.15.129:0/1347790306 shutdown_connections 2024-04-26T21:16:51.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.493+0000 7fb4a910e700 1 --2- 172.21.15.129:0/1347790306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a4107680 0x7fb4a410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.493+0000 7fb4a910e700 1 --2- 172.21.15.129:0/1347790306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4a41025a0 0x7fb4a4102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.493+0000 7fb4a910e700 1 --2- 172.21.15.129:0/1347790306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a4102f50 0x7fb4a4106fa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.493+0000 7fb4a910e700 1 -- 172.21.15.129:0/1347790306 >> 172.21.15.129:0/1347790306 conn(0x7fb4a40793c0 msgr2=0x7fb4a40797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.494+0000 7fb4a910e700 1 -- 172.21.15.129:0/1347790306 shutdown_connections 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.494+0000 7fb4a910e700 1 -- 172.21.15.129:0/1347790306 wait complete. 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.494+0000 7fb4a910e700 1 Processor -- start 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.494+0000 7fb4a910e700 1 -- start start 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a910e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4a41025a0 0x7fb4a419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a910e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a4102f50 0x7fb4a419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a910e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a4107680 0x7fb4a41a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a910e700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4112510 con 0x7fb4a4107680 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a910e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4112390 con 0x7fb4a41025a0 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a910e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4112690 con 0x7fb4a4102f50 2024-04-26T21:16:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a359e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a4107680 0x7fb4a41a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a4102f50 0x7fb4a419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a359e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a4107680 0x7fb4a41a59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39762/0 (socket says 172.21.15.129:39762) 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a359e700 1 -- 172.21.15.129:0/321384803 learned_addr learned my addr 172.21.15.129:0/321384803 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4a41025a0 0x7fb4a419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a359e700 1 -- 172.21.15.129:0/321384803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a4102f50 msgr2=0x7fb4a419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a359e700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a4102f50 0x7fb4a419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a359e700 1 -- 172.21.15.129:0/321384803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4a41025a0 msgr2=0x7fb4a419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a359e700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4a41025a0 0x7fb4a419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a359e700 1 -- 172.21.15.129:0/321384803 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a41a5ef0 con 0x7fb4a4107680 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a359e700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a4107680 0x7fb4a41a59b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb49c007ee0 tx=0x7fb49c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb48bfff700 1 -- 172.21.15.129:0/321384803 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c004bb0 con 0x7fb4a4107680 2024-04-26T21:16:51.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb48bfff700 1 -- 172.21.15.129:0/321384803 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49c004d50 con 0x7fb4a4107680 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a41a6120 con 0x7fb4a4107680 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.495+0000 7fb48bfff700 1 -- 172.21.15.129:0/321384803 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c005920 con 0x7fb4a4107680 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.496+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a41a6630 con 0x7fb4a4107680 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.496+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4a4063af0 con 0x7fb4a4107680 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.497+0000 7fb48bfff700 1 -- 172.21.15.129:0/321384803 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb49c01a020 con 0x7fb4a4107680 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.498+0000 7fb48bfff700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb48c0a3cd0 0x7fb48c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.498+0000 7fb4a2d9d700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb48c0a3cd0 0x7fb48c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.498+0000 7fb48bfff700 1 -- 172.21.15.129:0/321384803 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb49c0d0060 con 0x7fb4a4107680 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.498+0000 7fb48bfff700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] conn(0x7fb48c0ad4b0 0x7fb48c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.498+0000 7fb48bfff700 1 -- 172.21.15.129:0/321384803 --> [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb48c0aff80 con 0x7fb48c0ad4b0 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.498+0000 7fb48bfff700 1 -- 172.21.15.129:0/321384803 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fb49c0d0540 con 0x7fb4a4107680 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.498+0000 7fb4a2d9d700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb48c0a3cd0 0x7fb48c0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb494007be0 tx=0x7fb4940073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:51.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.499+0000 7fb4a259c700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] conn(0x7fb48c0ad4b0 0x7fb48c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.499+0000 7fb4a259c700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] conn(0x7fb48c0ad4b0 0x7fb48c0af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.500+0000 7fb48bfff700 1 -- 172.21.15.129:0/321384803 <== osd.11 v2:172.21.15.171:6824/2160214157 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb48c0aff80 con 0x7fb48c0ad4b0 2024-04-26T21:16:51.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.516+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 --> [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4a4063fa0 con 0x7fb48c0ad4b0 2024-04-26T21:16:51.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.516+0000 7fb48bfff700 1 -- 172.21.15.129:0/321384803 <== osd.11 v2:172.21.15.171:6824/2160214157 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4a4063fa0 con 0x7fb48c0ad4b0 2024-04-26T21:16:51.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 >> [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] conn(0x7fb48c0ad4b0 msgr2=0x7fb48c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] conn(0x7fb48c0ad4b0 0x7fb48c0af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb48c0a3cd0 msgr2=0x7fb48c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb48c0a3cd0 0x7fb48c0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb494007be0 tx=0x7fb4940073d0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a4107680 msgr2=0x7fb4a41a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 --2- 172.21.15.129:0/321384803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4a4107680 0x7fb4a41a59b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb49c007ee0 tx=0x7fb49c00d3b0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a359e700 1 -- 172.21.15.129:0/321384803 reap_dead start 2024-04-26T21:16:51.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 shutdown_connections 2024-04-26T21:16:51.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 >> 172.21.15.129:0/321384803 conn(0x7fb4a40793c0 msgr2=0x7fb4a4100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:51.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 shutdown_connections 2024-04-26T21:16:51.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:51.517+0000 7fb4a910e700 1 -- 172.21.15.129:0/321384803 wait complete. 2024-04-26T21:16:51.537 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:51.938 INFO:teuthology.orchestra.run.smithi129.stdout:257698037798 2024-04-26T21:16:51.938 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T21:16:52.396 INFO:teuthology.orchestra.run.smithi129.stdout:167503724598 2024-04-26T21:16:52.396 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T21:16:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:52 smithi150 ceph-mon[32290]: pgmap v314: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:52.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:52 smithi129 ceph-mon[29237]: pgmap v314: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:52.721 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:52.728 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:52.728 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:52.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:52 smithi171 ceph-mon[33478]: pgmap v314: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:54.632 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:54 smithi129 ceph-mon[29237]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:54 smithi150 ceph-mon[32290]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:54.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:54 smithi171 ceph-mon[33478]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T21:16:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-26T21:16:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-26T21:16:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-26T21:16:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-26T21:16:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-26T21:16:55.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-04-26T21:16:55.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]: dispatch 2024-04-26T21:16:55.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:55 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T21:16:55.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T21:16:55.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-26T21:16:55.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-26T21:16:55.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-26T21:16:55.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-26T21:16:55.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-26T21:16:55.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-04-26T21:16:55.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]: dispatch 2024-04-26T21:16:55.654 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:55 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T21:16:55.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T21:16:55.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-26T21:16:55.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-26T21:16:55.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-26T21:16:55.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-26T21:16:55.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-26T21:16:55.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-04-26T21:16:55.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]: dispatch 2024-04-26T21:16:55.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:55 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T21:16:56.124 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:56.125 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:56.125 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:56.363 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:56.580 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: pgmap v316: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:56.580 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T21:16:56.580 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-26T21:16:56.580 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-26T21:16:56.580 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-26T21:16:56.580 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-26T21:16:56.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-26T21:16:56.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-04-26T21:16:56.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]': finished 2024-04-26T21:16:56.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T21:16:56.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:56 smithi129 ceph-mon[29237]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: pgmap v316: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]': finished 2024-04-26T21:16:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T21:16:56.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:56 smithi150 ceph-mon[32290]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:16:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: pgmap v316: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]': finished 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T21:16:56.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:56 smithi171 ceph-mon[33478]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:16:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:57 smithi150 ceph-mon[32290]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:16:57.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:57 smithi129 ceph-mon[29237]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:16:57.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:57 smithi171 ceph-mon[33478]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:16:58.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.172+0000 7f6460fc4700 1 -- 172.21.15.129:0/1772175123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c102310 msgr2=0x7f645c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.172+0000 7f6460fc4700 1 --2- 172.21.15.129:0/1772175123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c102310 0x7f645c102790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f644c009990 tx=0x7f644c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.172+0000 7f6460fc4700 1 -- 172.21.15.129:0/1772175123 shutdown_connections 2024-04-26T21:16:58.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.172+0000 7f6460fc4700 1 --2- 172.21.15.129:0/1772175123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645c102cd0 0x7f645c10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.172+0000 7f6460fc4700 1 --2- 172.21.15.129:0/1772175123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f645c108380 0x7f645c108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.172+0000 7f6460fc4700 1 --2- 172.21.15.129:0/1772175123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c102310 0x7f645c102790 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.172+0000 7f6460fc4700 1 -- 172.21.15.129:0/1772175123 >> 172.21.15.129:0/1772175123 conn(0x7f645c079290 msgr2=0x7f645c0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:58.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.172+0000 7f6460fc4700 1 -- 172.21.15.129:0/1772175123 shutdown_connections 2024-04-26T21:16:58.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.172+0000 7f6460fc4700 1 -- 172.21.15.129:0/1772175123 wait complete. 2024-04-26T21:16:58.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f6460fc4700 1 Processor -- start 2024-04-26T21:16:58.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f6460fc4700 1 -- start start 2024-04-26T21:16:58.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f6460fc4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f645c102310 0x7f645c103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:58.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f6460fc4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645c102cd0 0x7f645c104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f6460fc4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c108380 0x7f645c104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f6460fc4700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1124a0 con 0x7f645c108380 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f6460fc4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f645c112320 con 0x7f645c102cd0 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f6460fc4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f645c112620 con 0x7f645c102310 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f645ad9d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c108380 0x7f645c104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f645ad9d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c108380 0x7f645c104a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54342/0 (socket says 172.21.15.129:54342) 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f645ad9d700 1 -- 172.21.15.129:0/3783674058 learned_addr learned my addr 172.21.15.129:0/3783674058 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f6459d9b700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645c102cd0 0x7f645c104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.173+0000 7f645a59c700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f645c102310 0x7f645c103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f645ad9d700 1 -- 172.21.15.129:0/3783674058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f645c102310 msgr2=0x7f645c103ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f645ad9d700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f645c102310 0x7f645c103ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f645ad9d700 1 -- 172.21.15.129:0/3783674058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645c102cd0 msgr2=0x7f645c104530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f645ad9d700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645c102cd0 0x7f645c104530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f645ad9d700 1 -- 172.21.15.129:0/3783674058 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c1a7fa0 con 0x7f645c108380 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f6459d9b700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645c102cd0 0x7f645c104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f645ad9d700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c108380 0x7f645c104a70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6454007ee0 tx=0x7f645400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f64537fe700 1 -- 172.21.15.129:0/3783674058 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645400ddd0 con 0x7f645c108380 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f64537fe700 1 -- 172.21.15.129:0/3783674058 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6454004020 con 0x7f645c108380 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f64537fe700 1 -- 172.21.15.129:0/3783674058 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454015660 con 0x7f645c108380 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c1a8290 con 0x7f645c108380 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.174+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c1a87d0 con 0x7f645c108380 2024-04-26T21:16:58.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.175+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c06a110 con 0x7f645c108380 2024-04-26T21:16:58.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.176+0000 7f64537fe700 1 -- 172.21.15.129:0/3783674058 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f64540044f0 con 0x7f645c108380 2024-04-26T21:16:58.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.177+0000 7f64537fe700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f64480a3d50 0x7f64480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:58.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.177+0000 7f645a59c700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f64480a3d50 0x7f64480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:58.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.177+0000 7f64537fe700 1 -- 172.21.15.129:0/3783674058 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f64540d0850 con 0x7f645c108380 2024-04-26T21:16:58.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.178+0000 7f645a59c700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f64480a3d50 0x7f64480a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6444005fd0 tx=0x7f6444005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:58.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.181+0000 7f64537fe700 1 -- 172.21.15.129:0/3783674058 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6454010070 con 0x7f645c108380 2024-04-26T21:16:58.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.390+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f645c1a8ad0 con 0x7f645c108380 2024-04-26T21:16:58.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.391+0000 7f64537fe700 1 -- 172.21.15.129:0/3783674058 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f645409d0c0 con 0x7f645c108380 2024-04-26T21:16:58.392 INFO:teuthology.orchestra.run.smithi129.stdout:146028888124 2024-04-26T21:16:58.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f64480a3d50 msgr2=0x7f64480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f64480a3d50 0x7f64480a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6444005fd0 tx=0x7f6444005db0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c108380 msgr2=0x7f645c104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c108380 0x7f645c104a70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6454007ee0 tx=0x7f645400d3b0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 shutdown_connections 2024-04-26T21:16:58.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f64480a3d50 0x7f64480a6210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f645c108380 0x7f645c104a70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f645c102310 0x7f645c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 --2- 172.21.15.129:0/3783674058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645c102cd0 0x7f645c104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 >> 172.21.15.129:0/3783674058 conn(0x7f645c079290 msgr2=0x7f645c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:58.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 shutdown_connections 2024-04-26T21:16:58.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.393+0000 7f6460fc4700 1 -- 172.21.15.129:0/3783674058 wait complete. 2024-04-26T21:16:58.436 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:58 smithi129 ceph-mon[29237]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-26T21:16:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:58 smithi150 ceph-mon[32290]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-26T21:16:58.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.693+0000 7f9417447700 1 -- 172.21.15.129:0/3487885286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410102320 msgr2=0x7f94101027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.693+0000 7f9417447700 1 --2- 172.21.15.129:0/3487885286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410102320 0x7f94101027a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f940c009990 tx=0x7f940c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.693+0000 7f9417447700 1 -- 172.21.15.129:0/3487885286 shutdown_connections 2024-04-26T21:16:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.693+0000 7f9417447700 1 --2- 172.21.15.129:0/3487885286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9410102ce0 0x7f941010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.693+0000 7f9417447700 1 --2- 172.21.15.129:0/3487885286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9410108390 0x7f9410108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.693+0000 7f9417447700 1 --2- 172.21.15.129:0/3487885286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410102320 0x7f94101027a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.693+0000 7f9417447700 1 -- 172.21.15.129:0/3487885286 >> 172.21.15.129:0/3487885286 conn(0x7f94100792d0 msgr2=0x7f94100796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.693+0000 7f9417447700 1 -- 172.21.15.129:0/3487885286 shutdown_connections 2024-04-26T21:16:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.693+0000 7f9417447700 1 -- 172.21.15.129:0/3487885286 wait complete. 2024-04-26T21:16:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f9417447700 1 Processor -- start 2024-04-26T21:16:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f9417447700 1 -- start start 2024-04-26T21:16:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f9417447700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9410102320 0x7f941019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f9417447700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9410102ce0 0x7f941019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f9417447700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410108390 0x7f94101a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f9417447700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9410112530 con 0x7f9410108390 2024-04-26T21:16:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f9417447700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f94101123b0 con 0x7f9410102320 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f9417447700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94101126b0 con 0x7f9410102ce0 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f94159e4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410108390 0x7f94101a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f94159e4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410108390 0x7f94101a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54368/0 (socket says 172.21.15.129:54368) 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f94159e4700 1 -- 172.21.15.129:0/77675178 learned_addr learned my addr 172.21.15.129:0/77675178 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f94151e3700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9410102320 0x7f941019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.694+0000 7f94149e2700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9410102ce0 0x7f941019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94159e4700 1 -- 172.21.15.129:0/77675178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9410102ce0 msgr2=0x7f941019af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94159e4700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9410102ce0 0x7f941019af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94159e4700 1 -- 172.21.15.129:0/77675178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9410102320 msgr2=0x7f941019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94159e4700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9410102320 0x7f941019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94159e4700 1 -- 172.21.15.129:0/77675178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94101a5ff0 con 0x7f9410108390 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94151e3700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9410102320 0x7f941019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94159e4700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410108390 0x7f94101a5ab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9408007d70 tx=0x7f940800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94067fc700 1 -- 172.21.15.129:0/77675178 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408016020 con 0x7f9410108390 2024-04-26T21:16:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94101a61c0 con 0x7f9410108390 2024-04-26T21:16:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94067fc700 1 -- 172.21.15.129:0/77675178 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f940800ad40 con 0x7f9410108390 2024-04-26T21:16:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94101a66d0 con 0x7f9410108390 2024-04-26T21:16:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.695+0000 7f94067fc700 1 -- 172.21.15.129:0/77675178 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94080115e0 con 0x7f9410108390 2024-04-26T21:16:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.696+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941010a900 con 0x7f9410108390 2024-04-26T21:16:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.698+0000 7f94067fc700 1 -- 172.21.15.129:0/77675178 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9408019020 con 0x7f9410108390 2024-04-26T21:16:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.698+0000 7f94067fc700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93fc0a3cd0 0x7f93fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.699+0000 7f94151e3700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93fc0a3cd0 0x7f93fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.699+0000 7f94067fc700 1 -- 172.21.15.129:0/77675178 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f94080d08d0 con 0x7f9410108390 2024-04-26T21:16:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.699+0000 7f94151e3700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93fc0a3cd0 0x7f93fc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f94000099d0 tx=0x7f9400008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.702+0000 7f94067fc700 1 -- 172.21.15.129:0/77675178 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9408020020 con 0x7f9410108390 2024-04-26T21:16:58.718 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:58.719 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:16:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:58 smithi171 ceph-mon[33478]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-26T21:16:58.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.931+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f941006b700 con 0x7f9410108390 2024-04-26T21:16:58.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.932+0000 7f94067fc700 1 -- 172.21.15.129:0/77675178 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f940809d250 con 0x7f9410108390 2024-04-26T21:16:58.933 INFO:teuthology.orchestra.run.smithi129.stdout:313532612637 2024-04-26T21:16:58.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93fc0a3cd0 msgr2=0x7f93fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93fc0a3cd0 0x7f93fc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f94000099d0 tx=0x7f9400008040 comp rx=0 tx=0).stop 2024-04-26T21:16:58.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410108390 msgr2=0x7f94101a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:58.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410108390 0x7f94101a5ab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9408007d70 tx=0x7f940800a340 comp rx=0 tx=0).stop 2024-04-26T21:16:58.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 shutdown_connections 2024-04-26T21:16:58.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f93fc0a3cd0 0x7f93fc0a6190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9410108390 0x7f94101a5ab0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9410102320 0x7f941019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 --2- 172.21.15.129:0/77675178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9410102ce0 0x7f941019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:58.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 >> 172.21.15.129:0/77675178 conn(0x7f94100792d0 msgr2=0x7f9410100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:58.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 shutdown_connections 2024-04-26T21:16:58.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.934+0000 7f9417447700 1 -- 172.21.15.129:0/77675178 wait complete. 2024-04-26T21:16:59.206 INFO:teuthology.orchestra.run.smithi129.stdout:347892350999 2024-04-26T21:16:59.206 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T21:16:59.440 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:59 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3783674058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:16:59.440 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:59 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3968922694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:16:59.440 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:16:59 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/77675178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:16:59.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 -- 172.21.15.129:0/207206145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006bbc0 msgr2=0x7f1d5006c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:59.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 --2- 172.21.15.129:0/207206145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006bbc0 0x7f1d5006c040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1d440099c0 tx=0x7f1d44009cd0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 -- 172.21.15.129:0/207206145 shutdown_connections 2024-04-26T21:16:59.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 --2- 172.21.15.129:0/207206145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5006c580 0x7f1d501102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 --2- 172.21.15.129:0/207206145 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d50109490 0x7f1d50109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 --2- 172.21.15.129:0/207206145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006bbc0 0x7f1d5006c040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 -- 172.21.15.129:0/207206145 >> 172.21.15.129:0/207206145 conn(0x7f1d50078390 msgr2=0x7f1d500787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:59.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 -- 172.21.15.129:0/207206145 shutdown_connections 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 -- 172.21.15.129:0/207206145 wait complete. 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.590+0000 7f1d566e0700 1 Processor -- start 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d566e0700 1 -- start start 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d566e0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d5006bbc0 0x7f1d5019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d566e0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006c580 0x7f1d5019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d566e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d50109490 0x7f1d50199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d566e0700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501135e0 con 0x7f1d5006c580 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d566e0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50113460 con 0x7f1d50109490 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d566e0700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50113760 con 0x7f1d5006bbc0 2024-04-26T21:16:59.524 INFO:teuthology.orchestra.run.smithi129.stdout:77309411399 2024-04-26T21:16:59.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006c580 0x7f1d5019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:59.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d54c7d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d50109490 0x7f1d50199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:59.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d4ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d5006bbc0 0x7f1d5019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:59.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d54c7d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d50109490 0x7f1d50199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:32830/0 (socket says 172.21.15.129:32830) 2024-04-26T21:16:59.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006c580 0x7f1d5019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54360/0 (socket says 172.21.15.129:54360) 2024-04-26T21:16:59.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.591+0000 7f1d54c7d700 1 -- 172.21.15.129:0/3968922694 learned_addr learned my addr 172.21.15.129:0/3968922694 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:16:59.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d54c7d700 1 -- 172.21.15.129:0/3968922694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d5006bbc0 msgr2=0x7f1d5019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:59.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d54c7d700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d5006bbc0 0x7f1d5019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d54c7d700 1 -- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006c580 msgr2=0x7f1d5019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d54c7d700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006c580 0x7f1d5019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d54c7d700 1 -- 172.21.15.129:0/3968922694 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5019a0a0 con 0x7f1d50109490 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d4f7fe700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006c580 0x7f1d5019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d54c7d700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d50109490 0x7f1d50199940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800ef30 tx=0x7f1d4800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d4d7fa700 1 -- 172.21.15.129:0/3968922694 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480092e0 con 0x7f1d50109490 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d4d7fa700 1 -- 172.21.15.129:0/3968922694 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d48004bb0 con 0x7f1d50109490 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d4d7fa700 1 -- 172.21.15.129:0/3968922694 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48004020 con 0x7f1d50109490 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5019a330 con 0x7f1d50109490 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.592+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d50102350 con 0x7f1d50109490 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.593+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d50104b80 con 0x7f1d50109490 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.594+0000 7f1d4d7fa700 1 -- 172.21.15.129:0/3968922694 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1d4801b030 con 0x7f1d50109490 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.595+0000 7f1d4d7fa700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1d380a3d50 0x7f1d380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.595+0000 7f1d4ffff700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1d380a3d50 0x7f1d380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:59.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.595+0000 7f1d4d7fa700 1 -- 172.21.15.129:0/3968922694 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f1d480d03a0 con 0x7f1d50109490 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.596+0000 7f1d4ffff700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1d380a3d50 0x7f1d380a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1d5019aa80 tx=0x7f1d4000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.599+0000 7f1d4d7fa700 1 -- 172.21.15.129:0/3968922694 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1d48022020 con 0x7f1d50109490 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.827+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1d5002e490 con 0x7f1d50109490 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.827+0000 7f1d4d7fa700 1 -- 172.21.15.129:0/3968922694 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d48010040 con 0x7f1d50109490 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1d380a3d50 msgr2=0x7f1d380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1d380a3d50 0x7f1d380a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1d5019aa80 tx=0x7f1d4000b3c0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d50109490 msgr2=0x7f1d50199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d50109490 0x7f1d50199940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800ef30 tx=0x7f1d4800c3a0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 shutdown_connections 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1d380a3d50 0x7f1d380a6210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d50109490 0x7f1d50199940 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d5006bbc0 0x7f1d5019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 --2- 172.21.15.129:0/3968922694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d5006c580 0x7f1d5019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 >> 172.21.15.129:0/3968922694 conn(0x7f1d50078390 msgr2=0x7f1d501010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:59.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 shutdown_connections 2024-04-26T21:16:59.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:16:58.831+0000 7f1d566e0700 1 -- 172.21.15.129:0/3968922694 wait complete. 2024-04-26T21:16:59.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:59 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3783674058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:16:59.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:59 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3968922694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:16:59.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:16:59 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/77675178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:16:59.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:59 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3783674058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:16:59.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:59 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3968922694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:16:59.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:16:59 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/77675178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:17:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:00 smithi150 ceph-mon[32290]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-26T21:17:00.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:00 smithi129 ceph-mon[29237]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-26T21:17:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:00 smithi171 ceph-mon[33478]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-26T21:17:01.188 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411399 for osd.2 2024-04-26T21:17:01.188 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:01.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:01 smithi129 ceph-mon[29237]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-04-26T21:17:01.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:01 smithi171 ceph-mon[33478]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-04-26T21:17:01.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:01 smithi150 ceph-mon[32290]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-04-26T21:17:01.954 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888120 got 146028888124 for osd.4 2024-04-26T21:17:01.954 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:03.867 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612634 got 313532612637 for osd.10 2024-04-26T21:17:03.867 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:04.523 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:04 smithi129 ceph-mon[29237]: pgmap v322: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-26T21:17:04.523 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:04 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:17:04.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:04 smithi150 ceph-mon[32290]: pgmap v322: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-26T21:17:04.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:04 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:17:04.655 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:04 smithi171 ceph-mon[33478]: pgmap v322: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-26T21:17:04.656 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:04 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:17:04.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.755+0000 7f85c11ac700 1 -- 172.21.15.129:0/920594978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc102f60 msgr2=0x7f85bc106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.755+0000 7f85c11ac700 1 --2- 172.21.15.129:0/920594978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc102f60 0x7f85bc106f60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85b000b0d0 tx=0x7f85b000b3e0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.755+0000 7f85c11ac700 1 -- 172.21.15.129:0/920594978 shutdown_connections 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.755+0000 7f85c11ac700 1 --2- 172.21.15.129:0/920594978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85bc107640 0x7f85bc10f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.755+0000 7f85c11ac700 1 --2- 172.21.15.129:0/920594978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85bc1025b0 0x7f85bc102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.755+0000 7f85c11ac700 1 --2- 172.21.15.129:0/920594978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc102f60 0x7f85bc106f60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.755+0000 7f85c11ac700 1 -- 172.21.15.129:0/920594978 >> 172.21.15.129:0/920594978 conn(0x7f85bc0792f0 msgr2=0x7f85bc079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.755+0000 7f85c11ac700 1 -- 172.21.15.129:0/920594978 shutdown_connections 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.756+0000 7f85c11ac700 1 -- 172.21.15.129:0/920594978 wait complete. 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.756+0000 7f85c11ac700 1 Processor -- start 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.756+0000 7f85c11ac700 1 -- start start 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85c11ac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85bc1025b0 0x7f85bc19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85c11ac700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85bc102f60 0x7f85bc19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:04.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85c11ac700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc107640 0x7f85bc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85c11ac700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc1124b0 con 0x7f85bc107640 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85c11ac700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc112330 con 0x7f85bc1025b0 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85c11ac700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc112630 con 0x7f85bc102f60 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bb59e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc107640 0x7f85bc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bb59e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc107640 0x7f85bc198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59676/0 (socket says 172.21.15.129:59676) 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bb59e700 1 -- 172.21.15.129:0/17122494 learned_addr learned my addr 172.21.15.129:0/17122494 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85ba59c700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85bc102f60 0x7f85bc19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bad9d700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85bc1025b0 0x7f85bc19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bb59e700 1 -- 172.21.15.129:0/17122494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85bc102f60 msgr2=0x7f85bc19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bb59e700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85bc102f60 0x7f85bc19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bb59e700 1 -- 172.21.15.129:0/17122494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85bc1025b0 msgr2=0x7f85bc19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bb59e700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85bc1025b0 0x7f85bc19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bb59e700 1 -- 172.21.15.129:0/17122494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc198ff0 con 0x7f85bc107640 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85bb59e700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc107640 0x7f85bc198890 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00d980 tx=0x7f85ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85a3fff700 1 -- 172.21.15.129:0/17122494 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac017070 con 0x7f85bc107640 2024-04-26T21:17:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85bc1992e0 con 0x7f85bc107640 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85a3fff700 1 -- 172.21.15.129:0/17122494 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85ac005020 con 0x7f85bc107640 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85a3fff700 1 -- 172.21.15.129:0/17122494 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00fa10 con 0x7f85bc107640 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.757+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85bc1a55a0 con 0x7f85bc107640 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.758+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85bc06a110 con 0x7f85bc107640 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.760+0000 7f85a3fff700 1 -- 172.21.15.129:0/17122494 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f85ac004a30 con 0x7f85bc107640 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.760+0000 7f85a3fff700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85a40a3cd0 0x7f85a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.760+0000 7f85bad9d700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85a40a3cd0 0x7f85a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.760+0000 7f85a3fff700 1 -- 172.21.15.129:0/17122494 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f85ac0d0780 con 0x7f85bc107640 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.761+0000 7f85bad9d700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85a40a3cd0 0x7f85a40a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85bc1999d0 tx=0x7f85b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.764+0000 7f85a3fff700 1 -- 172.21.15.129:0/17122494 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f85ac020020 con 0x7f85bc107640 2024-04-26T21:17:04.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.969+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f85bc02e490 con 0x7f85bc107640 2024-04-26T21:17:04.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.969+0000 7f85a3fff700 1 -- 172.21.15.129:0/17122494 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f85ac09cff0 con 0x7f85bc107640 2024-04-26T21:17:04.971 INFO:teuthology.orchestra.run.smithi129.stdout:201863462964 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85a40a3cd0 msgr2=0x7f85a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85a40a3cd0 0x7f85a40a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85bc1999d0 tx=0x7f85b4009260 comp rx=0 tx=0).stop 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc107640 msgr2=0x7f85bc198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc107640 0x7f85bc198890 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00d980 tx=0x7f85ac00dc90 comp rx=0 tx=0).stop 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 shutdown_connections 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f85a40a3cd0 0x7f85a40a6190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85bc107640 0x7f85bc198890 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85bc1025b0 0x7f85bc19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 --2- 172.21.15.129:0/17122494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85bc102f60 0x7f85bc19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 >> 172.21.15.129:0/17122494 conn(0x7f85bc0792f0 msgr2=0x7f85bc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 shutdown_connections 2024-04-26T21:17:04.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:04.972+0000 7f85c11ac700 1 -- 172.21.15.129:0/17122494 wait complete. 2024-04-26T21:17:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:05 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/17122494' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:17:05.657 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:05 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/17122494' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:17:05.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:05 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/17122494' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:17:06.620 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:06 smithi129 ceph-mon[29237]: pgmap v323: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s; 11 B/s, 0 objects/s recovering 2024-04-26T21:17:06.620 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:06 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.620 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:06 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.620 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:06 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1267119143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:17:06.620 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:06 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.620 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:06 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:06 smithi150 ceph-mon[32290]: pgmap v323: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s; 11 B/s, 0 objects/s recovering 2024-04-26T21:17:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:06 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:06 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:06 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1267119143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:17:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:06 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:06 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:06 smithi171 ceph-mon[33478]: pgmap v323: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s; 11 B/s, 0 objects/s recovering 2024-04-26T21:17:06.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:06 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:06 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:06 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1267119143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:17:06.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:06 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:06 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:06.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.438+0000 7f7a1c990700 1 -- 172.21.15.129:0/896689298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 msgr2=0x7f7a181102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:06.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.438+0000 7f7a1c990700 1 --2- 172.21.15.129:0/896689298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 0x7f7a181102c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000b0a0 tx=0x7f7a1000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.438+0000 7f7a1c990700 1 -- 172.21.15.129:0/896689298 shutdown_connections 2024-04-26T21:17:06.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.438+0000 7f7a1c990700 1 --2- 172.21.15.129:0/896689298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 0x7f7a181102c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.438+0000 7f7a1c990700 1 --2- 172.21.15.129:0/896689298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a181094f0 0x7f7a181098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.438+0000 7f7a1c990700 1 --2- 172.21.15.129:0/896689298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a18103470 0x7f7a181038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.438+0000 7f7a1c990700 1 -- 172.21.15.129:0/896689298 >> 172.21.15.129:0/896689298 conn(0x7f7a1807a5d0 msgr2=0x7f7a1807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:06.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.438+0000 7f7a1c990700 1 -- 172.21.15.129:0/896689298 shutdown_connections 2024-04-26T21:17:06.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.438+0000 7f7a1c990700 1 -- 172.21.15.129:0/896689298 wait complete. 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a1c990700 1 Processor -- start 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a1c990700 1 -- start start 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a1c990700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a18103470 0x7f7a1819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a1c990700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 0x7f7a1819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a1c990700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a181094f0 0x7f7a181999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a1c990700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18113610 con 0x7f7a18103e30 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a1c990700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18113490 con 0x7f7a181094f0 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a1c990700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18113790 con 0x7f7a18103470 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a15d9b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 0x7f7a1819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a15d9b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 0x7f7a1819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59688/0 (socket says 172.21.15.129:59688) 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a15d9b700 1 -- 172.21.15.129:0/1267119143 learned_addr learned my addr 172.21.15.129:0/1267119143 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a16d9d700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a181094f0 0x7f7a181999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a1659c700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a18103470 0x7f7a1819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stdout:257698037802 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a15d9b700 1 -- 172.21.15.129:0/1267119143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a18103470 msgr2=0x7f7a1819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a15d9b700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a18103470 0x7f7a1819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a15d9b700 1 -- 172.21.15.129:0/1267119143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a181094f0 msgr2=0x7f7a181999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a15d9b700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a181094f0 0x7f7a181999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a15d9b700 1 -- 172.21.15.129:0/1267119143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1819a110 con 0x7f7a18103e30 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.439+0000 7f7a15d9b700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 0x7f7a1819fd30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00d980 tx=0x7f7a0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.440+0000 7f7a077fe700 1 -- 172.21.15.129:0/1267119143 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c016070 con 0x7f7a18103e30 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.440+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1819a400 con 0x7f7a18103e30 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.440+0000 7f7a077fe700 1 -- 172.21.15.129:0/1267119143 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c004830 con 0x7f7a18103e30 2024-04-26T21:17:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.440+0000 7f7a077fe700 1 -- 172.21.15.129:0/1267119143 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c005540 con 0x7f7a18103e30 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.440+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a181a66c0 con 0x7f7a18103e30 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.441+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1806a110 con 0x7f7a18103e30 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.442+0000 7f7a077fe700 1 -- 172.21.15.129:0/1267119143 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7a0c004220 con 0x7f7a18103e30 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.442+0000 7f7a077fe700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7a000a3cd0 0x7f7a000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.442+0000 7f7a1659c700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7a000a3cd0 0x7f7a000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.442+0000 7f7a077fe700 1 -- 172.21.15.129:0/1267119143 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7a0c0d0330 con 0x7f7a18103e30 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.443+0000 7f7a1659c700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7a000a3cd0 0x7f7a000a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a1819aaf0 tx=0x7f7a08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.447+0000 7f7a077fe700 1 -- 172.21.15.129:0/1267119143 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7a0c01f020 con 0x7f7a18103e30 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.656+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7a1802e490 con 0x7f7a18103e30 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.657+0000 7f7a077fe700 1 -- 172.21.15.129:0/1267119143 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7a0c09cba0 con 0x7f7a18103e30 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7a000a3cd0 msgr2=0x7f7a000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:06.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7a000a3cd0 0x7f7a000a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a1819aaf0 tx=0x7f7a08009260 comp rx=0 tx=0).stop 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 msgr2=0x7f7a1819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 0x7f7a1819fd30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00d980 tx=0x7f7a0c00dc90 comp rx=0 tx=0).stop 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 shutdown_connections 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7a000a3cd0 0x7f7a000a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a181094f0 0x7f7a181999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a18103470 0x7f7a1819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 --2- 172.21.15.129:0/1267119143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a18103e30 0x7f7a1819fd30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 >> 172.21.15.129:0/1267119143 conn(0x7f7a1807a5d0 msgr2=0x7f7a18102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 shutdown_connections 2024-04-26T21:17:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:05.659+0000 7f7a1c990700 1 -- 172.21.15.129:0/1267119143 wait complete. 2024-04-26T21:17:08.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:08 smithi129 ceph-mon[29237]: pgmap v324: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 936 B/s rd, 1 op/s; 10 B/s, 0 objects/s recovering 2024-04-26T21:17:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:08 smithi150 ceph-mon[32290]: pgmap v324: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 936 B/s rd, 1 op/s; 10 B/s, 0 objects/s recovering 2024-04-26T21:17:08.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:08 smithi171 ceph-mon[33478]: pgmap v324: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 936 B/s rd, 1 op/s; 10 B/s, 0 objects/s recovering 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 -- 172.21.15.129:0/1302275339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2618109550 msgr2=0x7f2618109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 --2- 172.21.15.129:0/1302275339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2618109550 0x7f2618109930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f260c0099c0 tx=0x7f260c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 -- 172.21.15.129:0/1302275339 shutdown_connections 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 --2- 172.21.15.129:0/1302275339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2618103e70 0x7f2618110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 --2- 172.21.15.129:0/1302275339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2618109550 0x7f2618109930 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 --2- 172.21.15.129:0/1302275339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26181034b0 0x7f2618103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 -- 172.21.15.129:0/1302275339 >> 172.21.15.129:0/1302275339 conn(0x7f261807a610 msgr2=0x7f261807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 -- 172.21.15.129:0/1302275339 shutdown_connections 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 -- 172.21.15.129:0/1302275339 wait complete. 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 Processor -- start 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.340+0000 7f261f1e1700 1 -- start start 2024-04-26T21:17:09.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261f1e1700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26181034b0 0x7f2618075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261f1e1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2618103e70 0x7f2618075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261f1e1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2618109550 0x7f26180761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261f1e1700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2618113730 con 0x7f26181034b0 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261f1e1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26181135b0 con 0x7f2618109550 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261f1e1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f26181138b0 con 0x7f2618103e70 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261cf7d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26181034b0 0x7f2618075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261cf7d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26181034b0 0x7f2618075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59702/0 (socket says 172.21.15.129:59702) 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261cf7d700 1 -- 172.21.15.129:0/2525996792 learned_addr learned my addr 172.21.15.129:0/2525996792 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f2617fff700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2618103e70 0x7f2618075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261d77e700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2618109550 0x7f26180761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261cf7d700 1 -- 172.21.15.129:0/2525996792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2618103e70 msgr2=0x7f2618075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261cf7d700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2618103e70 0x7f2618075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261cf7d700 1 -- 172.21.15.129:0/2525996792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2618109550 msgr2=0x7f26180761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261cf7d700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2618109550 0x7f26180761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261cf7d700 1 -- 172.21.15.129:0/2525996792 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26181a81b0 con 0x7f26181034b0 2024-04-26T21:17:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f2617fff700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2618103e70 0x7f2618075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261cf7d700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26181034b0 0x7f2618075720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f260c00b580 tx=0x7f260c0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f2615ffb700 1 -- 172.21.15.129:0/2525996792 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c027070 con 0x7f26181034b0 2024-04-26T21:17:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26181a8440 con 0x7f26181034b0 2024-04-26T21:17:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f2615ffb700 1 -- 172.21.15.129:0/2525996792 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f260c004110 con 0x7f26181034b0 2024-04-26T21:17:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.341+0000 7f2615ffb700 1 -- 172.21.15.129:0/2525996792 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c00f460 con 0x7f26181034b0 2024-04-26T21:17:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.342+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26181a8890 con 0x7f26181034b0 2024-04-26T21:17:09.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.342+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261810b9e0 con 0x7f26181034b0 2024-04-26T21:17:09.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.344+0000 7f2615ffb700 1 -- 172.21.15.129:0/2525996792 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f260c02a070 con 0x7f26181034b0 2024-04-26T21:17:09.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.344+0000 7f2615ffb700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f26080a3cd0 0x7f26080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:09.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.344+0000 7f2617fff700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f26080a3cd0 0x7f26080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:09.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.344+0000 7f2615ffb700 1 -- 172.21.15.129:0/2525996792 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f260c0d15c0 con 0x7f26181034b0 2024-04-26T21:17:09.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.345+0000 7f2617fff700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f26080a3cd0 0x7f26080a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f261807ad30 tx=0x7f2604008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:09.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.349+0000 7f2615ffb700 1 -- 172.21.15.129:0/2525996792 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f260c05f060 con 0x7f26181034b0 2024-04-26T21:17:09.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.554+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2618076c50 con 0x7f26181034b0 2024-04-26T21:17:09.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.555+0000 7f2615ffb700 1 -- 172.21.15.129:0/2525996792 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f260c013070 con 0x7f26181034b0 2024-04-26T21:17:09.556 INFO:teuthology.orchestra.run.smithi129.stdout:55834574926 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f26080a3cd0 msgr2=0x7f26080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f26080a3cd0 0x7f26080a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f261807ad30 tx=0x7f2604008040 comp rx=0 tx=0).stop 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26181034b0 msgr2=0x7f2618075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26181034b0 0x7f2618075720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f260c00b580 tx=0x7f260c0037c0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 shutdown_connections 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f26080a3cd0 0x7f26080a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2618109550 0x7f26180761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26181034b0 0x7f2618075720 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 --2- 172.21.15.129:0/2525996792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2618103e70 0x7f2618075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:09.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 >> 172.21.15.129:0/2525996792 conn(0x7f261807a610 msgr2=0x7f2618102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:09.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 shutdown_connections 2024-04-26T21:17:09.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:09.557+0000 7f261f1e1700 1 -- 172.21.15.129:0/2525996792 wait complete. 2024-04-26T21:17:09.745 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037798 got 257698037802 for osd.8 2024-04-26T21:17:09.746 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:10.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:10 smithi129 ceph-mon[29237]: pgmap v325: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T21:17:10.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:10 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2525996792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:17:10.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:10 smithi150 ceph-mon[32290]: pgmap v325: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T21:17:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:10 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2525996792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:17:10.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.601+0000 7f6f7ac08700 1 -- 172.21.15.129:0/3889638530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 msgr2=0x7f6f741098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:10.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.601+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/3889638530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 0x7f6f741098a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009990 tx=0x7f6f68009ca0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.601+0000 7f6f7ac08700 1 -- 172.21.15.129:0/3889638530 shutdown_connections 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.601+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/3889638530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f74103e30 0x7f6f74110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.601+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/3889638530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 0x7f6f741098a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.601+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/3889638530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f74103470 0x7f6f741038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.601+0000 7f6f7ac08700 1 -- 172.21.15.129:0/3889638530 >> 172.21.15.129:0/3889638530 conn(0x7f6f7407a580 msgr2=0x7f6f7407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.601+0000 7f6f7ac08700 1 -- 172.21.15.129:0/3889638530 shutdown_connections 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.601+0000 7f6f7ac08700 1 -- 172.21.15.129:0/3889638530 wait complete. 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f7ac08700 1 Processor -- start 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f7ac08700 1 -- start start 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f7ac08700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f74103470 0x7f6f74075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f7ac08700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f74103e30 0x7f6f74075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f7ac08700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 0x7f6f740761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:10.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f7ac08700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6f741135f0 con 0x7f6f741094c0 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f7ac08700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74113470 con 0x7f6f74103470 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f7ac08700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74113770 con 0x7f6f74103e30 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f791a5700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 0x7f6f740761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f791a5700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 0x7f6f740761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59728/0 (socket says 172.21.15.129:59728) 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f791a5700 1 -- 172.21.15.129:0/749011343 learned_addr learned my addr 172.21.15.129:0/749011343 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f73fff700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f74103e30 0x7f6f74075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.602+0000 7f6f789a4700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f74103470 0x7f6f74075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f791a5700 1 -- 172.21.15.129:0/749011343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f74103e30 msgr2=0x7f6f74075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f791a5700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f74103e30 0x7f6f74075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f791a5700 1 -- 172.21.15.129:0/749011343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f74103470 msgr2=0x7f6f74075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f791a5700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f74103470 0x7f6f74075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f791a5700 1 -- 172.21.15.129:0/749011343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f741a80d0 con 0x7f6f741094c0 2024-04-26T21:17:10.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f791a5700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 0x7f6f740761a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00ada0 tx=0x7f6f6c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f71ffb700 1 -- 172.21.15.129:0/749011343 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c017020 con 0x7f6f741094c0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f741a83c0 con 0x7f6f741094c0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f71ffb700 1 -- 172.21.15.129:0/749011343 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f6c00ce30 con 0x7f6f741094c0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f71ffb700 1 -- 172.21.15.129:0/749011343 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c005350 con 0x7f6f741094c0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.603+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f741a8900 con 0x7f6f741094c0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.604+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7410b930 con 0x7f6f741094c0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.605+0000 7f6f71ffb700 1 -- 172.21.15.129:0/749011343 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6f6c00c980 con 0x7f6f741094c0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.605+0000 7f6f71ffb700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6f640a3cd0 0x7f6f640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.606+0000 7f6f789a4700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6f640a3cd0 0x7f6f640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.606+0000 7f6f71ffb700 1 -- 172.21.15.129:0/749011343 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6f6c0cfc90 con 0x7f6f741094c0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.606+0000 7f6f789a4700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6f640a3cd0 0x7f6f640a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f68006010 tx=0x7f6f6801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.610+0000 7f6f71ffb700 1 -- 172.21.15.129:0/749011343 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6f6c0d5020 con 0x7f6f741094c0 2024-04-26T21:17:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:10 smithi171 ceph-mon[33478]: pgmap v325: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T21:17:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:10 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2525996792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:17:10.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.814+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6f74076c50 con 0x7f6f741094c0 2024-04-26T21:17:10.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.815+0000 7f6f71ffb700 1 -- 172.21.15.129:0/749011343 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6f6c021020 con 0x7f6f741094c0 2024-04-26T21:17:10.816 INFO:teuthology.orchestra.run.smithi129.stdout:223338299439 2024-04-26T21:17:10.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6f640a3cd0 msgr2=0x7f6f640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:10.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6f640a3cd0 0x7f6f640a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f68006010 tx=0x7f6f6801a040 comp rx=0 tx=0).stop 2024-04-26T21:17:10.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 msgr2=0x7f6f740761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:10.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 0x7f6f740761a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00ada0 tx=0x7f6f6c00c330 comp rx=0 tx=0).stop 2024-04-26T21:17:10.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 shutdown_connections 2024-04-26T21:17:10.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6f640a3cd0 0x7f6f640a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f741094c0 0x7f6f740761a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f74103470 0x7f6f74075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 --2- 172.21.15.129:0/749011343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f74103e30 0x7f6f74075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:10.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.817+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 >> 172.21.15.129:0/749011343 conn(0x7f6f7407a580 msgr2=0x7f6f74102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:10.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.818+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 shutdown_connections 2024-04-26T21:17:10.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:10.818+0000 7f6f7ac08700 1 -- 172.21.15.129:0/749011343 wait complete. 2024-04-26T21:17:11.206 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462957 got 201863462964 for osd.6 2024-04-26T21:17:11.206 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:11.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:11 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/749011343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:17:11.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:11 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/749011343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:17:11.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:11 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/749011343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:17:11.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/622648442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102cd0 msgr2=0x7f1aa010f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:11.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/622648442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102cd0 0x7f1aa010f1c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800b0a0 tx=0x7f1a9800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:17:11.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/622648442 shutdown_connections 2024-04-26T21:17:11.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/622648442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102cd0 0x7f1aa010f1c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:11.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/622648442 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1aa0108380 0x7f1aa0108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:11.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/622648442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa0102310 0x7f1aa0102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:11.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/622648442 >> 172.21.15.129:0/622648442 conn(0x7f1aa0079290 msgr2=0x7f1aa00796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:11.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/622648442 shutdown_connections 2024-04-26T21:17:11.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/622648442 wait complete. 2024-04-26T21:17:11.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.968+0000 7f1aa5bc9700 1 Processor -- start 2024-04-26T21:17:11.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1aa5bc9700 1 -- start start 2024-04-26T21:17:11.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1aa5bc9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102310 0x7f1aa019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:11.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1aa5bc9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1aa0102cd0 0x7f1aa019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1aa5bc9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa0108380 0x7f1aa01a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1aa5bc9700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa01124c0 con 0x7f1aa0102310 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1aa5bc9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa0112340 con 0x7f1aa0108380 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1aa5bc9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa0112640 con 0x7f1aa0102cd0 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9f7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102310 0x7f1aa019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9f7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102310 0x7f1aa019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59742/0 (socket says 172.21.15.129:59742) 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9f7fe700 1 -- 172.21.15.129:0/258631810 learned_addr learned my addr 172.21.15.129:0/258631810 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9ffff700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa0108380 0x7f1aa01a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9effd700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1aa0102cd0 0x7f1aa019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9f7fe700 1 -- 172.21.15.129:0/258631810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1aa0102cd0 msgr2=0x7f1aa019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9f7fe700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1aa0102cd0 0x7f1aa019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9f7fe700 1 -- 172.21.15.129:0/258631810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa0108380 msgr2=0x7f1aa01a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9f7fe700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa0108380 0x7f1aa01a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9f7fe700 1 -- 172.21.15.129:0/258631810 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa01a5ea0 con 0x7f1aa0102310 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9effd700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1aa0102cd0 0x7f1aa019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:11.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.969+0000 7f1a9f7fe700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102310 0x7f1aa019a8e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e9c0 tx=0x7f1a9000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:11.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.970+0000 7f1a9cff9700 1 -- 172.21.15.129:0/258631810 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9000cbd0 con 0x7f1aa0102310 2024-04-26T21:17:11.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.970+0000 7f1a9cff9700 1 -- 172.21.15.129:0/258631810 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9000cd70 con 0x7f1aa0102310 2024-04-26T21:17:11.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.970+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa01a6070 con 0x7f1aa0102310 2024-04-26T21:17:11.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.970+0000 7f1a9cff9700 1 -- 172.21.15.129:0/258631810 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90005610 con 0x7f1aa0102310 2024-04-26T21:17:11.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.970+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa01a6560 con 0x7f1aa0102310 2024-04-26T21:17:11.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.970+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa006a110 con 0x7f1aa0102310 2024-04-26T21:17:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.972+0000 7f1a9cff9700 1 -- 172.21.15.129:0/258631810 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1a90004750 con 0x7f1aa0102310 2024-04-26T21:17:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.972+0000 7f1a9cff9700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1a8c0a3cd0 0x7f1a8c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.972+0000 7f1a9effd700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1a8c0a3cd0 0x7f1a8c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.972+0000 7f1a9cff9700 1 -- 172.21.15.129:0/258631810 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f1a900d0a40 con 0x7f1aa0102310 2024-04-26T21:17:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.973+0000 7f1a9effd700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1a8c0a3cd0 0x7f1a8c0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1a88005950 tx=0x7f1a880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:11.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:11.976+0000 7f1a9cff9700 1 -- 172.21.15.129:0/258631810 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1a90054060 con 0x7f1aa0102310 2024-04-26T21:17:12.054 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:12.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.193+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1aa006bf80 con 0x7f1aa0102310 2024-04-26T21:17:12.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.194+0000 7f1a9cff9700 1 -- 172.21.15.129:0/258631810 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1aa006bf80 con 0x7f1aa0102310 2024-04-26T21:17:12.195 INFO:teuthology.orchestra.run.smithi129.stdout:34359738451 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1a8c0a3cd0 msgr2=0x7f1a8c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1a8c0a3cd0 0x7f1a8c0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1a88005950 tx=0x7f1a880058e0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102310 msgr2=0x7f1aa019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102310 0x7f1aa019a8e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e9c0 tx=0x7f1a9000ecd0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 shutdown_connections 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1a8c0a3cd0 0x7f1a8c0a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa0108380 0x7f1aa01a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1aa0102310 0x7f1aa019a8e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 --2- 172.21.15.129:0/258631810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1aa0102cd0 0x7f1aa019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 >> 172.21.15.129:0/258631810 conn(0x7f1aa0079290 msgr2=0x7f1aa0100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:12.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 shutdown_connections 2024-04-26T21:17:12.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:12.196+0000 7f1aa5bc9700 1 -- 172.21.15.129:0/258631810 wait complete. 2024-04-26T21:17:12.535 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574926 for osd.1 2024-04-26T21:17:12.536 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:12 smithi150 ceph-mon[32290]: pgmap v326: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:17:12.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:12 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/258631810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:17:12.638 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:12 smithi129 ceph-mon[29237]: pgmap v326: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:17:12.638 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:12 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/258631810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:17:12.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:12 smithi171 ceph-mon[33478]: pgmap v326: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:17:12.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:12 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/258631810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:17:13.431 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299433 got 223338299439 for osd.7 2024-04-26T21:17:13.431 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:14.429 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:14 smithi129 ceph-mon[29237]: pgmap v327: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:14.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:14 smithi150 ceph-mon[32290]: pgmap v327: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:14 smithi171 ceph-mon[33478]: pgmap v327: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:14.840 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738451 for osd.0 2024-04-26T21:17:14.840 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:15.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.277+0000 7feaccf86700 1 -- 172.21.15.129:0/3499187672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac8103470 msgr2=0x7feac81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:15.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.277+0000 7feaccf86700 1 --2- 172.21.15.129:0/3499187672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac8103470 0x7feac81038f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feabc009990 tx=0x7feabc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.277+0000 7feaccf86700 1 -- 172.21.15.129:0/3499187672 shutdown_connections 2024-04-26T21:17:15.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.277+0000 7feaccf86700 1 --2- 172.21.15.129:0/3499187672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac8103e30 0x7feac8110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.277+0000 7feaccf86700 1 --2- 172.21.15.129:0/3499187672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac81094c0 0x7feac81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.277+0000 7feaccf86700 1 --2- 172.21.15.129:0/3499187672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac8103470 0x7feac81038f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.277+0000 7feaccf86700 1 -- 172.21.15.129:0/3499187672 >> 172.21.15.129:0/3499187672 conn(0x7feac807a580 msgr2=0x7feac807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:15.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.277+0000 7feaccf86700 1 -- 172.21.15.129:0/3499187672 shutdown_connections 2024-04-26T21:17:15.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.277+0000 7feaccf86700 1 -- 172.21.15.129:0/3499187672 wait complete. 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feaccf86700 1 Processor -- start 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feaccf86700 1 -- start start 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feaccf86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac8103470 0x7feac819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feaccf86700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac8103e30 0x7feac819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feaccf86700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac81094c0 0x7feac81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feaccf86700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feac8113610 con 0x7feac81094c0 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feaccf86700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feac8113490 con 0x7feac8103470 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feaccf86700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7feac8113790 con 0x7feac8103e30 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feac6d9d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac81094c0 0x7feac81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feac659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac8103470 0x7feac819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:15.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feac6d9d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac81094c0 0x7feac81999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52764/0 (socket says 172.21.15.129:52764) 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feac6d9d700 1 -- 172.21.15.129:0/644302998 learned_addr learned my addr 172.21.15.129:0/644302998 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.278+0000 7feac5d9b700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac8103e30 0x7feac819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feac6d9d700 1 -- 172.21.15.129:0/644302998 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac8103e30 msgr2=0x7feac819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feac6d9d700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac8103e30 0x7feac819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feac6d9d700 1 -- 172.21.15.129:0/644302998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac8103470 msgr2=0x7feac819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feac6d9d700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac8103470 0x7feac819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feac6d9d700 1 -- 172.21.15.129:0/644302998 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac819a110 con 0x7feac81094c0 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feac6d9d700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac81094c0 0x7feac81999b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feac0007ee0 tx=0x7feac000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feab77fe700 1 -- 172.21.15.129:0/644302998 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac000dd20 con 0x7feac81094c0 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feab77fe700 1 -- 172.21.15.129:0/644302998 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac0004020 con 0x7feac81094c0 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac819a400 con 0x7feac81094c0 2024-04-26T21:17:15.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feab77fe700 1 -- 172.21.15.129:0/644302998 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0015660 con 0x7feac81094c0 2024-04-26T21:17:15.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.279+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac81a66c0 con 0x7feac81094c0 2024-04-26T21:17:15.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.280+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac806a110 con 0x7feac81094c0 2024-04-26T21:17:15.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.281+0000 7feab77fe700 1 -- 172.21.15.129:0/644302998 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7feac00041d0 con 0x7feac81094c0 2024-04-26T21:17:15.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.282+0000 7feab77fe700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7feab00a3cd0 0x7feab00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:15.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.282+0000 7feac659c700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7feab00a3cd0 0x7feab00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:15.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.282+0000 7feab77fe700 1 -- 172.21.15.129:0/644302998 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7feac00cff20 con 0x7feac81094c0 2024-04-26T21:17:15.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.283+0000 7feac659c700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7feab00a3cd0 0x7feab00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feac819aaf0 tx=0x7feab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.287+0000 7feab77fe700 1 -- 172.21.15.129:0/644302998 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7feac0010070 con 0x7feac81094c0 2024-04-26T21:17:15.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.494+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7feac802e490 con 0x7feac81094c0 2024-04-26T21:17:15.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.494+0000 7feab77fe700 1 -- 172.21.15.129:0/644302998 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feac009c790 con 0x7feac81094c0 2024-04-26T21:17:15.495 INFO:teuthology.orchestra.run.smithi129.stdout:292057776166 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.496+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7feab00a3cd0 msgr2=0x7feab00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.496+0000 7feaccf86700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7feab00a3cd0 0x7feab00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feac819aaf0 tx=0x7feab8009260 comp rx=0 tx=0).stop 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.496+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac81094c0 msgr2=0x7feac81999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.496+0000 7feaccf86700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac81094c0 0x7feac81999b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feac0007ee0 tx=0x7feac000d3b0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.497+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 shutdown_connections 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.497+0000 7feaccf86700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7feab00a3cd0 0x7feab00a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.497+0000 7feaccf86700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feac81094c0 0x7feac81999b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.497+0000 7feaccf86700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac8103470 0x7feac819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.497+0000 7feaccf86700 1 --2- 172.21.15.129:0/644302998 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac8103e30 0x7feac819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.497+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 >> 172.21.15.129:0/644302998 conn(0x7feac807a580 msgr2=0x7feac81021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:15.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.497+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 shutdown_connections 2024-04-26T21:17:15.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.497+0000 7feaccf86700 1 -- 172.21.15.129:0/644302998 wait complete. 2024-04-26T21:17:16.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.829+0000 7f8136b6f700 1 -- 172.21.15.129:0/1020504534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130102320 msgr2=0x7f81301027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:16.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.829+0000 7f8136b6f700 1 --2- 172.21.15.129:0/1020504534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130102320 0x7f81301027a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8120009990 tx=0x7f8120009ca0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.829+0000 7f8136b6f700 1 -- 172.21.15.129:0/1020504534 shutdown_connections 2024-04-26T21:17:16.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.829+0000 7f8136b6f700 1 --2- 172.21.15.129:0/1020504534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8130102ce0 0x7f813010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.829+0000 7f8136b6f700 1 --2- 172.21.15.129:0/1020504534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8130108390 0x7f8130108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.829+0000 7f8136b6f700 1 --2- 172.21.15.129:0/1020504534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130102320 0x7f81301027a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.829+0000 7f8136b6f700 1 -- 172.21.15.129:0/1020504534 >> 172.21.15.129:0/1020504534 conn(0x7f81300792d0 msgr2=0x7f81300796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.829+0000 7f8136b6f700 1 -- 172.21.15.129:0/1020504534 shutdown_connections 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.829+0000 7f8136b6f700 1 -- 172.21.15.129:0/1020504534 wait complete. 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f8136b6f700 1 Processor -- start 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f8136b6f700 1 -- start start 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f8136b6f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8130102320 0x7f813019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f8136b6f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8130102ce0 0x7f813019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f8136b6f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130108390 0x7f8130198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f8136b6f700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8130112580 con 0x7f8130108390 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f8136b6f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8130112400 con 0x7f8130102ce0 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f8136b6f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8130112700 con 0x7f8130102320 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f813510c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130108390 0x7f8130198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f813510c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130108390 0x7f8130198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52782/0 (socket says 172.21.15.129:52782) 2024-04-26T21:17:16.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f813510c700 1 -- 172.21.15.129:0/2908260988 learned_addr learned my addr 172.21.15.129:0/2908260988 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:16.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f812ffff700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8130102ce0 0x7f813019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:16.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f813490b700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8130102320 0x7f813019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:16.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f813510c700 1 -- 172.21.15.129:0/2908260988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8130102320 msgr2=0x7f813019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:16.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f813510c700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8130102320 0x7f813019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f813510c700 1 -- 172.21.15.129:0/2908260988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8130102ce0 msgr2=0x7f813019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:16.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.830+0000 7f813510c700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8130102ce0 0x7f813019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.159 INFO:teuthology.orchestra.run.smithi129.stdout:111669149764 2024-04-26T21:17:16.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.831+0000 7f813510c700 1 -- 172.21.15.129:0/2908260988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81301990c0 con 0x7f8130108390 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.831+0000 7f812ffff700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8130102ce0 0x7f813019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.831+0000 7f813490b700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8130102320 0x7f813019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.831+0000 7f813510c700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130108390 0x7f8130198960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f811800b550 tx=0x7f811800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.831+0000 7f812dffb700 1 -- 172.21.15.129:0/2908260988 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118015020 con 0x7f8130108390 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.831+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301993b0 con 0x7f8130108390 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.831+0000 7f812dffb700 1 -- 172.21.15.129:0/2908260988 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8118011ce0 con 0x7f8130108390 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.831+0000 7f812dffb700 1 -- 172.21.15.129:0/2908260988 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118010d40 con 0x7f8130108390 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.831+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813006b100 con 0x7f8130108390 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.832+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f813006a110 con 0x7f8130108390 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.833+0000 7f812dffb700 1 -- 172.21.15.129:0/2908260988 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8118018020 con 0x7f8130108390 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.833+0000 7f812dffb700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f811c0a5ee0 0x7f811c0a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.834+0000 7f813490b700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f811c0a5ee0 0x7f811c0a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.834+0000 7f812dffb700 1 -- 172.21.15.129:0/2908260988 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f81180cfc70 con 0x7f8130108390 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.834+0000 7f813490b700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f811c0a5ee0 0x7f811c0a83a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8128007fd0 tx=0x7f8128007830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:15.838+0000 7f812dffb700 1 -- 172.21.15.129:0/2908260988 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f811801f020 con 0x7f8130108390 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.043+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8130199e40 con 0x7f8130108390 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.043+0000 7f812dffb700 1 -- 172.21.15.129:0/2908260988 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f811809c5f0 con 0x7f8130108390 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f811c0a5ee0 msgr2=0x7f811c0a83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f811c0a5ee0 0x7f811c0a83a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8128007fd0 tx=0x7f8128007830 comp rx=0 tx=0).stop 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130108390 msgr2=0x7f8130198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130108390 0x7f8130198960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f811800b550 tx=0x7f811800b910 comp rx=0 tx=0).stop 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 shutdown_connections 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f811c0a5ee0 0x7f811c0a83a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8130108390 0x7f8130198960 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8130102320 0x7f813019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 --2- 172.21.15.129:0/2908260988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8130102ce0 0x7f813019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 >> 172.21.15.129:0/2908260988 conn(0x7f81300792d0 msgr2=0x7f8130100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 shutdown_connections 2024-04-26T21:17:16.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:16.045+0000 7f8136b6f700 1 -- 172.21.15.129:0/2908260988 wait complete. 2024-04-26T21:17:16.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:16 smithi129 ceph-mon[29237]: pgmap v328: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:16.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:16 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/644302998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:17:16.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:16 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2908260988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:17:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:16 smithi150 ceph-mon[32290]: pgmap v328: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:16 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/644302998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:17:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:16 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2908260988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:17:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:16 smithi171 ceph-mon[33478]: pgmap v328: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:16 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/644302998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:17:16.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:16 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2908260988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:17:17.312 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149759 got 111669149764 for osd.3 2024-04-26T21:17:17.313 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:18.140 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776166 for osd.9 2024-04-26T21:17:18.140 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:18 smithi150 ceph-mon[32290]: pgmap v329: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:18.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:18 smithi129 ceph-mon[29237]: pgmap v329: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:18.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:18 smithi171 ceph-mon[33478]: pgmap v329: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:19.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.325+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/4093147464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0f8101ac0 msgr2=0x7fd0f8101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:19.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.325+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/4093147464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0f8101ac0 0x7fd0f8101f40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009990 tx=0x7fd0e0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.325+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/4093147464 shutdown_connections 2024-04-26T21:17:19.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.325+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/4093147464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0f8102480 0x7fd0f810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.325+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/4093147464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0f8107ac0 0x7fd0f8107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.325+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/4093147464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0f8101ac0 0x7fd0f8101f40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.325+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/4093147464 >> 172.21.15.129:0/4093147464 conn(0x7fd0f8078920 msgr2=0x7fd0f80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:19.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/4093147464 shutdown_connections 2024-04-26T21:17:19.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/4093147464 wait complete. 2024-04-26T21:17:19.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 Processor -- start 2024-04-26T21:17:19.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 -- start start 2024-04-26T21:17:19.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0f8101ac0 0x7fd0f819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:19.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0f8102480 0x7fd0f819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:19.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0f8107ac0 0x7fd0f8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:19.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8111430 con 0x7fd0f8101ac0 2024-04-26T21:17:19.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81112b0 con 0x7fd0f8107ac0 2024-04-26T21:17:19.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.326+0000 7fd0fdc8d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81115b0 con 0x7fd0f8102480 2024-04-26T21:17:19.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0effff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0f8102480 0x7fd0f819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0effff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0f8102480 0x7fd0f819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54392/0 (socket says 172.21.15.129:54392) 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0effff700 1 -- 172.21.15.129:0/3121944024 learned_addr learned my addr 172.21.15.129:0/3121944024 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0f77fe700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0f8101ac0 0x7fd0f819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0f7fff700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0f8107ac0 0x7fd0f8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0effff700 1 -- 172.21.15.129:0/3121944024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0f8107ac0 msgr2=0x7fd0f8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0effff700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0f8107ac0 0x7fd0f8197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0effff700 1 -- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0f8101ac0 msgr2=0x7fd0f819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0effff700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0f8101ac0 0x7fd0f819d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0effff700 1 -- 172.21.15.129:0/3121944024 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f8197f80 con 0x7fd0f8102480 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.327+0000 7fd0effff700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0f8102480 0x7fd0f819db70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0005f50 tx=0x7fd0e00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.328+0000 7fd0f57fa700 1 -- 172.21.15.129:0/3121944024 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0003c70 con 0x7fd0f8102480 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.328+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f8198210 con 0x7fd0f8102480 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.328+0000 7fd0f57fa700 1 -- 172.21.15.129:0/3121944024 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e0020460 con 0x7fd0f8102480 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.328+0000 7fd0f57fa700 1 -- 172.21.15.129:0/3121944024 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e000f6c0 con 0x7fd0f8102480 2024-04-26T21:17:19.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.328+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f806ce40 con 0x7fd0f8102480 2024-04-26T21:17:19.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.329+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f8103200 con 0x7fd0f8102480 2024-04-26T21:17:19.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.330+0000 7fd0f57fa700 1 -- 172.21.15.129:0/3121944024 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd0e0028020 con 0x7fd0f8102480 2024-04-26T21:17:19.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.330+0000 7fd0f57fa700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0e40a3cd0 0x7fd0e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:19.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.331+0000 7fd0f77fe700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0e40a3cd0 0x7fd0e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:19.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.331+0000 7fd0f57fa700 1 -- 172.21.15.129:0/3121944024 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd0e00d5820 con 0x7fd0f8102480 2024-04-26T21:17:19.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.331+0000 7fd0f77fe700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0e40a3cd0 0x7fd0e40a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800bc00 tx=0x7fd0e800b490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.335+0000 7fd0f57fa700 1 -- 172.21.15.129:0/3121944024 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd0e0013070 con 0x7fd0f8102480 2024-04-26T21:17:19.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.542+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd0f8198c30 con 0x7fd0f8102480 2024-04-26T21:17:19.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.543+0000 7fd0f57fa700 1 -- 172.21.15.129:0/3121944024 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd0e00a2120 con 0x7fd0f8102480 2024-04-26T21:17:19.544 INFO:teuthology.orchestra.run.smithi129.stdout:167503724604 2024-04-26T21:17:19.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0e40a3cd0 msgr2=0x7fd0e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0e40a3cd0 0x7fd0e40a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800bc00 tx=0x7fd0e800b490 comp rx=0 tx=0).stop 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0f8102480 msgr2=0x7fd0f819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0f8102480 0x7fd0f819db70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0005f50 tx=0x7fd0e00047b0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 shutdown_connections 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0e40a3cd0 0x7fd0e40a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0f8107ac0 0x7fd0f8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0f8101ac0 0x7fd0f819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 --2- 172.21.15.129:0/3121944024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0f8102480 0x7fd0f819db70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 >> 172.21.15.129:0/3121944024 conn(0x7fd0f8078920 msgr2=0x7fd0f8104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 shutdown_connections 2024-04-26T21:17:19.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:19.545+0000 7fd0fdc8d700 1 -- 172.21.15.129:0/3121944024 wait complete. 2024-04-26T21:17:20.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.359+0000 7f1e4b649700 1 -- 172.21.15.129:0/577986543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 msgr2=0x7f1e441087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.359+0000 7f1e4b649700 1 --2- 172.21.15.129:0/577986543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 0x7f1e441087b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009990 tx=0x7f1e40009ca0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.359+0000 7f1e4b649700 1 -- 172.21.15.129:0/577986543 shutdown_connections 2024-04-26T21:17:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.359+0000 7f1e4b649700 1 --2- 172.21.15.129:0/577986543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e44102d10 0x7f1e4410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.359+0000 7f1e4b649700 1 --2- 172.21.15.129:0/577986543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 0x7f1e441087b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.359+0000 7f1e4b649700 1 --2- 172.21.15.129:0/577986543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e44102350 0x7f1e441027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.359+0000 7f1e4b649700 1 -- 172.21.15.129:0/577986543 >> 172.21.15.129:0/577986543 conn(0x7f1e44078390 msgr2=0x7f1e440787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.359+0000 7f1e4b649700 1 -- 172.21.15.129:0/577986543 shutdown_connections 2024-04-26T21:17:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.359+0000 7f1e4b649700 1 -- 172.21.15.129:0/577986543 wait complete. 2024-04-26T21:17:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e4b649700 1 Processor -- start 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e4b649700 1 -- start start 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e4b649700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e44102350 0x7f1e4419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e4b649700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e44102d10 0x7f1e4419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e4b649700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 0x7f1e44198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e4b649700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441124c0 con 0x7f1e441083d0 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e4b649700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44112340 con 0x7f1e44102d10 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e4b649700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44112640 con 0x7f1e44102350 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e49be6700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 0x7f1e44198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e49be6700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 0x7f1e44198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52814/0 (socket says 172.21.15.129:52814) 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e493e5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e44102350 0x7f1e4419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.360+0000 7f1e49be6700 1 -- 172.21.15.129:0/2630751743 learned_addr learned my addr 172.21.15.129:0/2630751743 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e48be4700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e44102d10 0x7f1e4419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e49be6700 1 -- 172.21.15.129:0/2630751743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e44102350 msgr2=0x7f1e4419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:20.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e49be6700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e44102350 0x7f1e4419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e49be6700 1 -- 172.21.15.129:0/2630751743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e44102d10 msgr2=0x7f1e4419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:20.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e49be6700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e44102d10 0x7f1e4419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e49be6700 1 -- 172.21.15.129:0/2630751743 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e44198fe0 con 0x7f1e441083d0 2024-04-26T21:17:20.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e49be6700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 0x7f1e44198800 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00efb0 tx=0x7f1e3c00c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:20.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e3a7fc700 1 -- 172.21.15.129:0/2630751743 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c011670 con 0x7f1e441083d0 2024-04-26T21:17:20.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e3a7fc700 1 -- 172.21.15.129:0/2630751743 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c00ce20 con 0x7f1e441083d0 2024-04-26T21:17:20.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e441992d0 con 0x7f1e441083d0 2024-04-26T21:17:20.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.361+0000 7f1e3a7fc700 1 -- 172.21.15.129:0/2630751743 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c01abe0 con 0x7f1e441083d0 2024-04-26T21:17:20.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.362+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4406bb90 con 0x7f1e441083d0 2024-04-26T21:17:20.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.363+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e44103a40 con 0x7f1e441083d0 2024-04-26T21:17:20.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.364+0000 7f1e3a7fc700 1 -- 172.21.15.129:0/2630751743 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1e3c011810 con 0x7f1e441083d0 2024-04-26T21:17:20.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.364+0000 7f1e3a7fc700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1e300a3cd0 0x7f1e300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:20.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.365+0000 7f1e493e5700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1e300a3cd0 0x7f1e300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:20.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.365+0000 7f1e3a7fc700 1 -- 172.21.15.129:0/2630751743 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f1e3c0d2de0 con 0x7f1e441083d0 2024-04-26T21:17:20.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.365+0000 7f1e493e5700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1e300a3cd0 0x7f1e300a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000b5c0 tx=0x7f1e40005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:20.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.371+0000 7f1e3a7fc700 1 -- 172.21.15.129:0/2630751743 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1e3c016030 con 0x7f1e441083d0 2024-04-26T21:17:20.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.581+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f1e4406c620 con 0x7f1e441083d0 2024-04-26T21:17:20.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.581+0000 7f1e3a7fc700 1 -- 172.21.15.129:0/2630751743 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1e3c09f650 con 0x7f1e441083d0 2024-04-26T21:17:20.583 INFO:teuthology.orchestra.run.smithi129.stdout:347892351004 2024-04-26T21:17:20.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1e300a3cd0 msgr2=0x7f1e300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1e300a3cd0 0x7f1e300a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000b5c0 tx=0x7f1e40005960 comp rx=0 tx=0).stop 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 msgr2=0x7f1e44198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 0x7f1e44198800 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00efb0 tx=0x7f1e3c00c550 comp rx=0 tx=0).stop 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 shutdown_connections 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1e300a3cd0 0x7f1e300a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e441083d0 0x7f1e44198800 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e44102350 0x7f1e4419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 --2- 172.21.15.129:0/2630751743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e44102d10 0x7f1e4419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.583+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 >> 172.21.15.129:0/2630751743 conn(0x7f1e44078390 msgr2=0x7f1e441010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.584+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 shutdown_connections 2024-04-26T21:17:20.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:20.584+0000 7f1e4b649700 1 -- 172.21.15.129:0/2630751743 wait complete. 2024-04-26T21:17:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:20 smithi150 ceph-mon[32290]: pgmap v330: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:20 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3121944024' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:17:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:20 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:20 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:20.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:20 smithi129 ceph-mon[29237]: pgmap v330: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:20.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:20 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3121944024' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:17:20.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:20 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:20.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:20 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:20 smithi171 ceph-mon[33478]: pgmap v330: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:20 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3121944024' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:17:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:20 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:20 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:21.382 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724598 got 167503724604 for osd.5 2024-04-26T21:17:21.382 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:21.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:17:21.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:17:21.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:21.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:21 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2630751743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:17:21.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:17:21.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:17:21.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:21.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:21 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2630751743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:17:21.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:17:21.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:17:21.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:17:21.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:21 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2630751743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:17:21.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350999 got 347892351004 for osd.11 2024-04-26T21:17:21.905 DEBUG:teuthology.parallel:result is None 2024-04-26T21:17:21.906 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T21:17:21.906 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:17:22.149 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:22.535 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:22 smithi129 ceph-mon[29237]: pgmap v331: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:22.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:22 smithi171 ceph-mon[33478]: pgmap v331: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:22.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:22 smithi150 ceph-mon[32290]: pgmap v331: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:23.618 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:23 smithi129 ceph-mon[29237]: pgmap v332: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:23 smithi171 ceph-mon[33478]: pgmap v332: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:23.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.876+0000 7f9783dac700 1 -- 172.21.15.129:0/4135267091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 msgr2=0x7f977c1029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:23.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.876+0000 7f9783dac700 1 --2- 172.21.15.129:0/4135267091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 0x7f977c1029f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97780099c0 tx=0x7f9778009cd0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.876+0000 7f9783dac700 1 -- 172.21.15.129:0/4135267091 shutdown_connections 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.876+0000 7f9783dac700 1 --2- 172.21.15.129:0/4135267091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f977c107660 0x7f977c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.876+0000 7f9783dac700 1 --2- 172.21.15.129:0/4135267091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 0x7f977c1029f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.876+0000 7f9783dac700 1 --2- 172.21.15.129:0/4135267091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f977c102fc0 0x7f977c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.876+0000 7f9783dac700 1 -- 172.21.15.129:0/4135267091 >> 172.21.15.129:0/4135267091 conn(0x7f977c078370 msgr2=0x7f977c078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.876+0000 7f9783dac700 1 -- 172.21.15.129:0/4135267091 shutdown_connections 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.876+0000 7f9783dac700 1 -- 172.21.15.129:0/4135267091 wait complete. 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9783dac700 1 Processor -- start 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9783dac700 1 -- start start 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9783dac700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 0x7f977c1092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9783dac700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f977c102fc0 0x7f977c109810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:23.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9783dac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f977c107660 0x7f977c109d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9783dac700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f977c112590 con 0x7f977c102610 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9783dac700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f977c112410 con 0x7f977c107660 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9783dac700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f977c112710 con 0x7f977c102fc0 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9781b48700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 0x7f977c1092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9781b48700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 0x7f977c1092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48648/0 (socket says 172.21.15.129:48648) 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9781b48700 1 -- 172.21.15.129:0/3367013055 learned_addr learned my addr 172.21.15.129:0/3367013055 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9782349700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f977c107660 0x7f977c109d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9781b48700 1 -- 172.21.15.129:0/3367013055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f977c102fc0 msgr2=0x7f977c109810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9781347700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f977c102fc0 0x7f977c109810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9781b48700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f977c102fc0 0x7f977c109810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9781b48700 1 -- 172.21.15.129:0/3367013055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f977c107660 msgr2=0x7f977c109d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:23.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9781b48700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f977c107660 0x7f977c109d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9781b48700 1 -- 172.21.15.129:0/3367013055 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c1a80b0 con 0x7f977c102610 2024-04-26T21:17:23.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.877+0000 7f9782349700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f977c107660 0x7f977c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:23.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.878+0000 7f9781b48700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 0x7f977c1092d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f977800b580 tx=0x7f97780037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:23.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.878+0000 7f9772ffd700 1 -- 172.21.15.129:0/3367013055 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778027070 con 0x7f977c102610 2024-04-26T21:17:23.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.878+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1a8340 con 0x7f977c102610 2024-04-26T21:17:23.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.878+0000 7f9772ffd700 1 -- 172.21.15.129:0/3367013055 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9778004110 con 0x7f977c102610 2024-04-26T21:17:23.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.878+0000 7f9772ffd700 1 -- 172.21.15.129:0/3367013055 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977800f460 con 0x7f977c102610 2024-04-26T21:17:23.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.878+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c1a8790 con 0x7f977c102610 2024-04-26T21:17:23.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.879+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c06a110 con 0x7f977c102610 2024-04-26T21:17:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:23 smithi150 ceph-mon[32290]: pgmap v332: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:23.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.880+0000 7f9772ffd700 1 -- 172.21.15.129:0/3367013055 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f977802a070 con 0x7f977c102610 2024-04-26T21:17:23.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.880+0000 7f9772ffd700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f97680a3cd0 0x7f97680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:23.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.880+0000 7f9781347700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f97680a3cd0 0x7f97680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:23.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.880+0000 7f9772ffd700 1 -- 172.21.15.129:0/3367013055 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f97780d1990 con 0x7f977c102610 2024-04-26T21:17:23.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.881+0000 7f9781347700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f97680a3cd0 0x7f97680a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f977c108900 tx=0x7f976c005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:23.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:23.884+0000 7f9772ffd700 1 -- 172.21.15.129:0/3367013055 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9778031070 con 0x7f977c102610 2024-04-26T21:17:24.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.080+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f977c101b00 con 0x7f97680a3cd0 2024-04-26T21:17:24.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.087+0000 7f9772ffd700 1 -- 172.21.15.129:0/3367013055 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7f977c101b00 con 0x7f97680a3cd0 2024-04-26T21:17:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:24.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f97680a3cd0 msgr2=0x7f97680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:24.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f97680a3cd0 0x7f97680a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f977c108900 tx=0x7f976c005cf0 comp rx=0 tx=0).stop 2024-04-26T21:17:24.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 msgr2=0x7f977c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 0x7f977c1092d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f977800b580 tx=0x7f97780037c0 comp rx=0 tx=0).stop 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 shutdown_connections 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f97680a3cd0 0x7f97680a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f977c107660 0x7f977c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f977c102610 0x7f977c1092d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 --2- 172.21.15.129:0/3367013055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f977c102fc0 0x7f977c109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 >> 172.21.15.129:0/3367013055 conn(0x7f977c078370 msgr2=0x7f977c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 shutdown_connections 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:24.090+0000 7f9783dac700 1 -- 172.21.15.129:0/3367013055 wait complete. 2024-04-26T21:17:24.095 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-26T21:17:24.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:24 smithi129 ceph-mon[29237]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:17:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:24 smithi171 ceph-mon[33478]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:17:24.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:24 smithi150 ceph-mon[32290]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:17:24.888 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-04-26T21:17:23.297169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643932,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":289,"num_read_kb":244,"num_write":257,"num_write_kb":3014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360704,"kb_used_data":11484,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581120,"statfs":{"total":1151940427776,"available":1151571066880,"internally_reserved":0,"allocated":11759616,"data_stored":7437312,"data_compressed":182064,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003535"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359578+0000","last_change":"2024-04-26T21:16:03.700581+0000","last_active":"2024-04-26T21:16:56.359578+0000","last_peered":"2024-04-26T21:16:56.359578+0000","last_clean":"2024-04-26T21:16:56.359578+0000","last_became_active":"2024-04-26T21:16:03.700280+0000","last_became_peered":"2024-04-26T21:16:03.700280+0000","last_unstale":"2024-04-26T21:16:56.359578+0000","last_undegraded":"2024-04-26T21:16:56.359578+0000","last_fullsized":"2024-04-26T21:16:56.359578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:01.329784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368025+0000","last_change":"2024-04-26T21:16:03.702058+0000","last_active":"2024-04-26T21:16:56.368025+0000","last_peered":"2024-04-26T21:16:56.368025+0000","last_clean":"2024-04-26T21:16:56.368025+0000","last_became_active":"2024-04-26T21:16:03.701728+0000","last_became_peered":"2024-04-26T21:16:03.701728+0000","last_unstale":"2024-04-26T21:16:56.368025+0000","last_undegraded":"2024-04-26T21:16:56.368025+0000","last_fullsized":"2024-04-26T21:16:56.368025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:12.718225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367712+0000","last_change":"2024-04-26T21:16:03.710461+0000","last_active":"2024-04-26T21:16:56.367712+0000","last_peered":"2024-04-26T21:16:56.367712+0000","last_clean":"2024-04-26T21:16:56.367712+0000","last_became_active":"2024-04-26T21:16:03.710130+0000","last_became_peered":"2024-04-26T21:16:03.710130+0000","last_unstale":"2024-04-26T21:16:56.367712+0000","last_undegraded":"2024-04-26T21:16:56.367712+0000","last_fullsized":"2024-04-26T21:16:56.367712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:21.294677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360172+0000","last_change":"2024-04-26T21:16:03.706836+0000","last_active":"2024-04-26T21:16:56.360172+0000","last_peered":"2024-04-26T21:16:56.360172+0000","last_clean":"2024-04-26T21:16:56.360172+0000","last_became_active":"2024-04-26T21:16:03.706515+0000","last_became_peered":"2024-04-26T21:16:03.706515+0000","last_unstale":"2024-04-26T21:16:56.360172+0000","last_undegraded":"2024-04-26T21:16:56.360172+0000","last_fullsized":"2024-04-26T21:16:56.360172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:56.198337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362204+0000","last_change":"2024-04-26T21:16:03.704865+0000","last_active":"2024-04-26T21:16:56.362204+0000","last_peered":"2024-04-26T21:16:56.362204+0000","last_clean":"2024-04-26T21:16:56.362204+0000","last_became_active":"2024-04-26T21:16:03.704560+0000","last_became_peered":"2024-04-26T21:16:03.704560+0000","last_unstale":"2024-04-26T21:16:56.362204+0000","last_undegraded":"2024-04-26T21:16:56.362204+0000","last_fullsized":"2024-04-26T21:16:56.362204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:51:03.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712989+0000","last_change":"2024-04-26T21:16:03.700265+0000","last_active":"2024-04-26T21:16:56.712989+0000","last_peered":"2024-04-26T21:16:56.712989+0000","last_clean":"2024-04-26T21:16:56.712989+0000","last_became_active":"2024-04-26T21:16:03.699878+0000","last_became_peered":"2024-04-26T21:16:03.699878+0000","last_unstale":"2024-04-26T21:16:56.712989+0000","last_undegraded":"2024-04-26T21:16:56.712989+0000","last_fullsized":"2024-04-26T21:16:56.712989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:12.907254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361631+0000","last_change":"2024-04-26T21:16:03.697569+0000","last_active":"2024-04-26T21:16:56.361631+0000","last_peered":"2024-04-26T21:16:56.361631+0000","last_clean":"2024-04-26T21:16:56.361631+0000","last_became_active":"2024-04-26T21:16:03.697262+0000","last_became_peered":"2024-04-26T21:16:03.697262+0000","last_unstale":"2024-04-26T21:16:56.361631+0000","last_undegraded":"2024-04-26T21:16:56.361631+0000","last_fullsized":"2024-04-26T21:16:56.361631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:28.645750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363218+0000","last_change":"2024-04-26T21:16:03.705788+0000","last_active":"2024-04-26T21:16:56.363218+0000","last_peered":"2024-04-26T21:16:56.363218+0000","last_clean":"2024-04-26T21:16:56.363218+0000","last_became_active":"2024-04-26T21:16:03.705479+0000","last_became_peered":"2024-04-26T21:16:03.705479+0000","last_unstale":"2024-04-26T21:16:56.363218+0000","last_undegraded":"2024-04-26T21:16:56.363218+0000","last_fullsized":"2024-04-26T21:16:56.363218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:27.306777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713655+0000","last_change":"2024-04-26T21:16:03.709716+0000","last_active":"2024-04-26T21:16:56.713655+0000","last_peered":"2024-04-26T21:16:56.713655+0000","last_clean":"2024-04-26T21:16:56.713655+0000","last_became_active":"2024-04-26T21:16:03.709219+0000","last_became_peered":"2024-04-26T21:16:03.709219+0000","last_unstale":"2024-04-26T21:16:56.713655+0000","last_undegraded":"2024-04-26T21:16:56.713655+0000","last_fullsized":"2024-04-26T21:16:56.713655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:18.936075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713179+0000","last_change":"2024-04-26T21:16:03.694745+0000","last_active":"2024-04-26T21:16:56.713179+0000","last_peered":"2024-04-26T21:16:56.713179+0000","last_clean":"2024-04-26T21:16:56.713179+0000","last_became_active":"2024-04-26T21:16:03.694044+0000","last_became_peered":"2024-04-26T21:16:03.694044+0000","last_unstale":"2024-04-26T21:16:56.713179+0000","last_undegraded":"2024-04-26T21:16:56.713179+0000","last_fullsized":"2024-04-26T21:16:56.713179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:13.060467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362033+0000","last_change":"2024-04-26T21:16:03.706434+0000","last_active":"2024-04-26T21:16:56.362033+0000","last_peered":"2024-04-26T21:16:56.362033+0000","last_clean":"2024-04-26T21:16:56.362033+0000","last_became_active":"2024-04-26T21:16:03.706093+0000","last_became_peered":"2024-04-26T21:16:03.706093+0000","last_unstale":"2024-04-26T21:16:56.362033+0000","last_undegraded":"2024-04-26T21:16:56.362033+0000","last_fullsized":"2024-04-26T21:16:56.362033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:04.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360084+0000","last_change":"2024-04-26T21:16:03.707672+0000","last_active":"2024-04-26T21:16:56.360084+0000","last_peered":"2024-04-26T21:16:56.360084+0000","last_clean":"2024-04-26T21:16:56.360084+0000","last_became_active":"2024-04-26T21:16:03.707344+0000","last_became_peered":"2024-04-26T21:16:03.707344+0000","last_unstale":"2024-04-26T21:16:56.360084+0000","last_undegraded":"2024-04-26T21:16:56.360084+0000","last_fullsized":"2024-04-26T21:16:56.360084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:07.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361281+0000","last_change":"2024-04-26T21:16:03.699816+0000","last_active":"2024-04-26T21:16:56.361281+0000","last_peered":"2024-04-26T21:16:56.361281+0000","last_clean":"2024-04-26T21:16:56.361281+0000","last_became_active":"2024-04-26T21:16:03.699514+0000","last_became_peered":"2024-04-26T21:16:03.699514+0000","last_unstale":"2024-04-26T21:16:56.361281+0000","last_undegraded":"2024-04-26T21:16:56.361281+0000","last_fullsized":"2024-04-26T21:16:56.361281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:59.230438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367152+0000","last_change":"2024-04-26T21:16:03.709627+0000","last_active":"2024-04-26T21:16:56.367152+0000","last_peered":"2024-04-26T21:16:56.367152+0000","last_clean":"2024-04-26T21:16:56.367152+0000","last_became_active":"2024-04-26T21:16:03.709238+0000","last_became_peered":"2024-04-26T21:16:03.709238+0000","last_unstale":"2024-04-26T21:16:56.367152+0000","last_undegraded":"2024-04-26T21:16:56.367152+0000","last_fullsized":"2024-04-26T21:16:56.367152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:55.303059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.366111+0000","last_change":"2024-04-26T21:16:03.702275+0000","last_active":"2024-04-26T21:16:56.366111+0000","last_peered":"2024-04-26T21:16:56.366111+0000","last_clean":"2024-04-26T21:16:56.366111+0000","last_became_active":"2024-04-26T21:16:03.701966+0000","last_became_peered":"2024-04-26T21:16:03.701966+0000","last_unstale":"2024-04-26T21:16:56.366111+0000","last_undegraded":"2024-04-26T21:16:56.366111+0000","last_fullsized":"2024-04-26T21:16:56.366111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:02.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713281+0000","last_change":"2024-04-26T21:16:03.711979+0000","last_active":"2024-04-26T21:16:56.713281+0000","last_peered":"2024-04-26T21:16:56.713281+0000","last_clean":"2024-04-26T21:16:56.713281+0000","last_became_active":"2024-04-26T21:16:03.711637+0000","last_became_peered":"2024-04-26T21:16:03.711637+0000","last_unstale":"2024-04-26T21:16:56.713281+0000","last_undegraded":"2024-04-26T21:16:56.713281+0000","last_fullsized":"2024-04-26T21:16:56.713281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:27.817993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359639+0000","last_change":"2024-04-26T21:16:03.701303+0000","last_active":"2024-04-26T21:16:56.359639+0000","last_peered":"2024-04-26T21:16:56.359639+0000","last_clean":"2024-04-26T21:16:56.359639+0000","last_became_active":"2024-04-26T21:16:03.701001+0000","last_became_peered":"2024-04-26T21:16:03.701001+0000","last_unstale":"2024-04-26T21:16:56.359639+0000","last_undegraded":"2024-04-26T21:16:56.359639+0000","last_fullsized":"2024-04-26T21:16:56.359639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:08.349915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362455+0000","last_change":"2024-04-26T21:16:03.696775+0000","last_active":"2024-04-26T21:16:56.362455+0000","last_peered":"2024-04-26T21:16:56.362455+0000","last_clean":"2024-04-26T21:16:56.362455+0000","last_became_active":"2024-04-26T21:16:03.696433+0000","last_became_peered":"2024-04-26T21:16:03.696433+0000","last_unstale":"2024-04-26T21:16:56.362455+0000","last_undegraded":"2024-04-26T21:16:56.362455+0000","last_fullsized":"2024-04-26T21:16:56.362455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:50.029169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361906+0000","last_change":"2024-04-26T21:16:03.698188+0000","last_active":"2024-04-26T21:16:56.361906+0000","last_peered":"2024-04-26T21:16:56.361906+0000","last_clean":"2024-04-26T21:16:56.361906+0000","last_became_active":"2024-04-26T21:16:03.697882+0000","last_became_peered":"2024-04-26T21:16:03.697882+0000","last_unstale":"2024-04-26T21:16:56.361906+0000","last_undegraded":"2024-04-26T21:16:56.361906+0000","last_fullsized":"2024-04-26T21:16:56.361906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:04.737619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362709+0000","last_change":"2024-04-26T21:16:03.275296+0000","last_active":"2024-04-26T21:16:56.362709+0000","last_peered":"2024-04-26T21:16:56.362709+0000","last_clean":"2024-04-26T21:16:56.362709+0000","last_became_active":"2024-04-26T21:16:03.274733+0000","last_became_peered":"2024-04-26T21:16:03.274733+0000","last_unstale":"2024-04-26T21:16:56.362709+0000","last_undegraded":"2024-04-26T21:16:56.362709+0000","last_fullsized":"2024-04-26T21:16:56.362709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:16.512934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362172+0000","last_change":"2024-04-26T21:16:03.705823+0000","last_active":"2024-04-26T21:16:56.362172+0000","last_peered":"2024-04-26T21:16:56.362172+0000","last_clean":"2024-04-26T21:16:56.362172+0000","last_became_active":"2024-04-26T21:16:03.705512+0000","last_became_peered":"2024-04-26T21:16:03.705512+0000","last_unstale":"2024-04-26T21:16:56.362172+0000","last_undegraded":"2024-04-26T21:16:56.362172+0000","last_fullsized":"2024-04-26T21:16:56.362172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:17.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364297+0000","last_change":"2024-04-26T21:16:03.285008+0000","last_active":"2024-04-26T21:16:56.364297+0000","last_peered":"2024-04-26T21:16:56.364297+0000","last_clean":"2024-04-26T21:16:56.364297+0000","last_became_active":"2024-04-26T21:16:03.284679+0000","last_became_peered":"2024-04-26T21:16:03.284679+0000","last_unstale":"2024-04-26T21:16:56.364297+0000","last_undegraded":"2024-04-26T21:16:56.364297+0000","last_fullsized":"2024-04-26T21:16:56.364297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:35.404086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368858+0000","last_change":"2024-04-26T21:16:03.710571+0000","last_active":"2024-04-26T21:16:56.368858+0000","last_peered":"2024-04-26T21:16:56.368858+0000","last_clean":"2024-04-26T21:16:56.368858+0000","last_became_active":"2024-04-26T21:16:03.710237+0000","last_became_peered":"2024-04-26T21:16:03.710237+0000","last_unstale":"2024-04-26T21:16:56.368858+0000","last_undegraded":"2024-04-26T21:16:56.368858+0000","last_fullsized":"2024-04-26T21:16:56.368858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:53.188554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369209+0000","last_change":"2024-04-26T21:16:56.368746+0000","last_active":"2024-04-26T21:16:56.369209+0000","last_peered":"2024-04-26T21:16:56.369209+0000","last_clean":"2024-04-26T21:16:56.369209+0000","last_became_active":"2024-04-26T21:16:56.365062+0000","last_became_peered":"2024-04-26T21:16:56.365062+0000","last_unstale":"2024-04-26T21:16:56.369209+0000","last_undegraded":"2024-04-26T21:16:56.369209+0000","last_fullsized":"2024-04-26T21:16:56.369209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:36.165146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713534+0000","last_change":"2024-04-26T21:16:03.711355+0000","last_active":"2024-04-26T21:16:56.713534+0000","last_peered":"2024-04-26T21:16:56.713534+0000","last_clean":"2024-04-26T21:16:56.713534+0000","last_became_active":"2024-04-26T21:16:03.710885+0000","last_became_peered":"2024-04-26T21:16:03.710885+0000","last_unstale":"2024-04-26T21:16:56.713534+0000","last_undegraded":"2024-04-26T21:16:56.713534+0000","last_fullsized":"2024-04-26T21:16:56.713534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:10.515590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362768+0000","last_change":"2024-04-26T21:16:03.278484+0000","last_active":"2024-04-26T21:16:56.362768+0000","last_peered":"2024-04-26T21:16:56.362768+0000","last_clean":"2024-04-26T21:16:56.362768+0000","last_became_active":"2024-04-26T21:16:03.277402+0000","last_became_peered":"2024-04-26T21:16:03.277402+0000","last_unstale":"2024-04-26T21:16:56.362768+0000","last_undegraded":"2024-04-26T21:16:56.362768+0000","last_fullsized":"2024-04-26T21:16:56.362768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:21.807942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714671+0000","last_change":"2024-04-26T21:16:56.714671+0000","last_active":"2024-04-26T21:16:56.714671+0000","last_peered":"2024-04-26T21:16:56.714671+0000","last_clean":"2024-04-26T21:16:56.714671+0000","last_became_active":"2024-04-26T21:16:56.714209+0000","last_became_peered":"2024-04-26T21:16:56.714209+0000","last_unstale":"2024-04-26T21:16:56.714671+0000","last_undegraded":"2024-04-26T21:16:56.714671+0000","last_fullsized":"2024-04-26T21:16:56.714671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:44.790915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363442+0000","last_change":"2024-04-26T21:16:03.277219+0000","last_active":"2024-04-26T21:16:56.363442+0000","last_peered":"2024-04-26T21:16:56.363442+0000","last_clean":"2024-04-26T21:16:56.363442+0000","last_became_active":"2024-04-26T21:16:03.276885+0000","last_became_peered":"2024-04-26T21:16:03.276885+0000","last_unstale":"2024-04-26T21:16:56.363442+0000","last_undegraded":"2024-04-26T21:16:56.363442+0000","last_fullsized":"2024-04-26T21:16:56.363442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:45.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362963+0000","last_change":"2024-04-26T21:16:03.703453+0000","last_active":"2024-04-26T21:16:56.362963+0000","last_peered":"2024-04-26T21:16:56.362963+0000","last_clean":"2024-04-26T21:16:56.362963+0000","last_became_active":"2024-04-26T21:16:03.703161+0000","last_became_peered":"2024-04-26T21:16:03.703161+0000","last_unstale":"2024-04-26T21:16:56.362963+0000","last_undegraded":"2024-04-26T21:16:56.362963+0000","last_fullsized":"2024-04-26T21:16:56.362963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.999243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712837+0000","last_change":"2024-04-26T21:16:03.274815+0000","last_active":"2024-04-26T21:16:56.712837+0000","last_peered":"2024-04-26T21:16:56.712837+0000","last_clean":"2024-04-26T21:16:56.712837+0000","last_became_active":"2024-04-26T21:16:03.274509+0000","last_became_peered":"2024-04-26T21:16:03.274509+0000","last_unstale":"2024-04-26T21:16:56.712837+0000","last_undegraded":"2024-04-26T21:16:56.712837+0000","last_fullsized":"2024-04-26T21:16:56.712837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:27.212714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.371100+0000","last_change":"2024-04-26T21:16:56.371100+0000","last_active":"2024-04-26T21:16:56.371100+0000","last_peered":"2024-04-26T21:16:56.371100+0000","last_clean":"2024-04-26T21:16:56.371100+0000","last_became_active":"2024-04-26T21:16:56.370711+0000","last_became_peered":"2024-04-26T21:16:56.370711+0000","last_unstale":"2024-04-26T21:16:56.371100+0000","last_undegraded":"2024-04-26T21:16:56.371100+0000","last_fullsized":"2024-04-26T21:16:56.371100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:00.143815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360698+0000","last_change":"2024-04-26T21:16:03.270417+0000","last_active":"2024-04-26T21:16:56.360698+0000","last_peered":"2024-04-26T21:16:56.360698+0000","last_clean":"2024-04-26T21:16:56.360698+0000","last_became_active":"2024-04-26T21:16:03.270066+0000","last_became_peered":"2024-04-26T21:16:03.270066+0000","last_unstale":"2024-04-26T21:16:56.360698+0000","last_undegraded":"2024-04-26T21:16:56.360698+0000","last_fullsized":"2024-04-26T21:16:56.360698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:46.794698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363117+0000","last_change":"2024-04-26T21:16:03.700221+0000","last_active":"2024-04-26T21:16:56.363117+0000","last_peered":"2024-04-26T21:16:56.363117+0000","last_clean":"2024-04-26T21:16:56.363117+0000","last_became_active":"2024-04-26T21:16:03.699798+0000","last_became_peered":"2024-04-26T21:16:03.699798+0000","last_unstale":"2024-04-26T21:16:56.363117+0000","last_undegraded":"2024-04-26T21:16:56.363117+0000","last_fullsized":"2024-04-26T21:16:56.363117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:52.083700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362074+0000","last_change":"2024-04-26T21:16:03.282355+0000","last_active":"2024-04-26T21:16:56.362074+0000","last_peered":"2024-04-26T21:16:56.362074+0000","last_clean":"2024-04-26T21:16:56.362074+0000","last_became_active":"2024-04-26T21:16:03.281169+0000","last_became_peered":"2024-04-26T21:16:03.281169+0000","last_unstale":"2024-04-26T21:16:56.362074+0000","last_undegraded":"2024-04-26T21:16:56.362074+0000","last_fullsized":"2024-04-26T21:16:56.362074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:55.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714327+0000","last_change":"2024-04-26T21:16:56.714327+0000","last_active":"2024-04-26T21:16:56.714327+0000","last_peered":"2024-04-26T21:16:56.714327+0000","last_clean":"2024-04-26T21:16:56.714327+0000","last_became_active":"2024-04-26T21:16:56.713967+0000","last_became_peered":"2024-04-26T21:16:56.713967+0000","last_unstale":"2024-04-26T21:16:56.714327+0000","last_undegraded":"2024-04-26T21:16:56.714327+0000","last_fullsized":"2024-04-26T21:16:56.714327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:32.475044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713294+0000","last_change":"2024-04-26T21:16:03.284948+0000","last_active":"2024-04-26T21:16:56.713294+0000","last_peered":"2024-04-26T21:16:56.713294+0000","last_clean":"2024-04-26T21:16:56.713294+0000","last_became_active":"2024-04-26T21:16:03.284570+0000","last_became_peered":"2024-04-26T21:16:03.284570+0000","last_unstale":"2024-04-26T21:16:56.713294+0000","last_undegraded":"2024-04-26T21:16:56.713294+0000","last_fullsized":"2024-04-26T21:16:56.713294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:22.883834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713580+0000","last_change":"2024-04-26T21:16:03.710359+0000","last_active":"2024-04-26T21:16:56.713580+0000","last_peered":"2024-04-26T21:16:56.713580+0000","last_clean":"2024-04-26T21:16:56.713580+0000","last_became_active":"2024-04-26T21:16:03.710053+0000","last_became_peered":"2024-04-26T21:16:03.710053+0000","last_unstale":"2024-04-26T21:16:56.713580+0000","last_undegraded":"2024-04-26T21:16:56.713580+0000","last_fullsized":"2024-04-26T21:16:56.713580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:45.981083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361874+0000","last_change":"2024-04-26T21:16:03.278787+0000","last_active":"2024-04-26T21:16:56.361874+0000","last_peered":"2024-04-26T21:16:56.361874+0000","last_clean":"2024-04-26T21:16:56.361874+0000","last_became_active":"2024-04-26T21:16:03.278483+0000","last_became_peered":"2024-04-26T21:16:03.278483+0000","last_unstale":"2024-04-26T21:16:56.361874+0000","last_undegraded":"2024-04-26T21:16:56.361874+0000","last_fullsized":"2024-04-26T21:16:56.361874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:40.303372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372769+0000","last_change":"2024-04-26T21:16:56.372769+0000","last_active":"2024-04-26T21:16:56.372769+0000","last_peered":"2024-04-26T21:16:56.372769+0000","last_clean":"2024-04-26T21:16:56.372769+0000","last_became_active":"2024-04-26T21:16:56.372393+0000","last_became_peered":"2024-04-26T21:16:56.372393+0000","last_unstale":"2024-04-26T21:16:56.372769+0000","last_undegraded":"2024-04-26T21:16:56.372769+0000","last_fullsized":"2024-04-26T21:16:56.372769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:29.818635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364174+0000","last_change":"2024-04-26T21:16:03.277387+0000","last_active":"2024-04-26T21:16:56.364174+0000","last_peered":"2024-04-26T21:16:56.364174+0000","last_clean":"2024-04-26T21:16:56.364174+0000","last_became_active":"2024-04-26T21:16:03.274765+0000","last_became_peered":"2024-04-26T21:16:03.274765+0000","last_unstale":"2024-04-26T21:16:56.364174+0000","last_undegraded":"2024-04-26T21:16:56.364174+0000","last_fullsized":"2024-04-26T21:16:56.364174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:51.081883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713473+0000","last_change":"2024-04-26T21:16:03.710977+0000","last_active":"2024-04-26T21:16:56.713473+0000","last_peered":"2024-04-26T21:16:56.713473+0000","last_clean":"2024-04-26T21:16:56.713473+0000","last_became_active":"2024-04-26T21:16:03.710284+0000","last_became_peered":"2024-04-26T21:16:03.710284+0000","last_unstale":"2024-04-26T21:16:56.713473+0000","last_undegraded":"2024-04-26T21:16:56.713473+0000","last_fullsized":"2024-04-26T21:16:56.713473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:46.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'25","reported_seq":201,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:17:21.803024+0000","last_change":"2024-04-26T21:16:03.276865+0000","last_active":"2024-04-26T21:17:21.803024+0000","last_peered":"2024-04-26T21:17:21.803024+0000","last_clean":"2024-04-26T21:17:21.803024+0000","last_became_active":"2024-04-26T21:16:03.276564+0000","last_became_peered":"2024-04-26T21:16:03.276564+0000","last_unstale":"2024-04-26T21:17:21.803024+0000","last_undegraded":"2024-04-26T21:17:21.803024+0000","last_fullsized":"2024-04-26T21:17:21.803024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:33.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52532,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":157,"num_read_kb":129,"num_write":25,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362088+0000","last_change":"2024-04-26T21:16:03.700441+0000","last_active":"2024-04-26T21:16:56.362088+0000","last_peered":"2024-04-26T21:16:56.362088+0000","last_clean":"2024-04-26T21:16:56.362088+0000","last_became_active":"2024-04-26T21:16:03.700142+0000","last_became_peered":"2024-04-26T21:16:03.700142+0000","last_unstale":"2024-04-26T21:16:56.362088+0000","last_undegraded":"2024-04-26T21:16:56.362088+0000","last_fullsized":"2024-04-26T21:16:56.362088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:35.166603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361872+0000","last_change":"2024-04-26T21:16:03.280144+0000","last_active":"2024-04-26T21:16:56.361872+0000","last_peered":"2024-04-26T21:16:56.361872+0000","last_clean":"2024-04-26T21:16:56.361872+0000","last_became_active":"2024-04-26T21:16:03.279844+0000","last_became_peered":"2024-04-26T21:16:03.279844+0000","last_unstale":"2024-04-26T21:16:56.361872+0000","last_undegraded":"2024-04-26T21:16:56.361872+0000","last_fullsized":"2024-04-26T21:16:56.361872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:00.704181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361493+0000","last_change":"2024-04-26T21:16:03.701732+0000","last_active":"2024-04-26T21:16:56.361493+0000","last_peered":"2024-04-26T21:16:56.361493+0000","last_clean":"2024-04-26T21:16:56.361493+0000","last_became_active":"2024-04-26T21:16:03.701424+0000","last_became_peered":"2024-04-26T21:16:03.701424+0000","last_unstale":"2024-04-26T21:16:56.361493+0000","last_undegraded":"2024-04-26T21:16:56.361493+0000","last_fullsized":"2024-04-26T21:16:56.361493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:02.293086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363077+0000","last_change":"2024-04-26T21:16:03.276816+0000","last_active":"2024-04-26T21:16:56.363077+0000","last_peered":"2024-04-26T21:16:56.363077+0000","last_clean":"2024-04-26T21:16:56.363077+0000","last_became_active":"2024-04-26T21:16:03.276471+0000","last_became_peered":"2024-04-26T21:16:03.276471+0000","last_unstale":"2024-04-26T21:16:56.363077+0000","last_undegraded":"2024-04-26T21:16:56.363077+0000","last_fullsized":"2024-04-26T21:16:56.363077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:44.630638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361866+0000","last_change":"2024-04-26T21:16:03.701002+0000","last_active":"2024-04-26T21:16:56.361866+0000","last_peered":"2024-04-26T21:16:56.361866+0000","last_clean":"2024-04-26T21:16:56.361866+0000","last_became_active":"2024-04-26T21:16:03.700649+0000","last_became_peered":"2024-04-26T21:16:03.700649+0000","last_unstale":"2024-04-26T21:16:56.361866+0000","last_undegraded":"2024-04-26T21:16:56.361866+0000","last_fullsized":"2024-04-26T21:16:56.361866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:45.571735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359679+0000","last_change":"2024-04-26T21:16:03.273727+0000","last_active":"2024-04-26T21:16:56.359679+0000","last_peered":"2024-04-26T21:16:56.359679+0000","last_clean":"2024-04-26T21:16:56.359679+0000","last_became_active":"2024-04-26T21:16:03.273287+0000","last_became_peered":"2024-04-26T21:16:03.273287+0000","last_unstale":"2024-04-26T21:16:56.359679+0000","last_undegraded":"2024-04-26T21:16:56.359679+0000","last_fullsized":"2024-04-26T21:16:56.359679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:48.713646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361559+0000","last_change":"2024-04-26T21:16:03.696760+0000","last_active":"2024-04-26T21:16:56.361559+0000","last_peered":"2024-04-26T21:16:56.361559+0000","last_clean":"2024-04-26T21:16:56.361559+0000","last_became_active":"2024-04-26T21:16:03.696446+0000","last_became_peered":"2024-04-26T21:16:03.696446+0000","last_unstale":"2024-04-26T21:16:56.361559+0000","last_undegraded":"2024-04-26T21:16:56.361559+0000","last_fullsized":"2024-04-26T21:16:56.361559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:30.710360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361962+0000","last_change":"2024-04-26T21:16:03.282643+0000","last_active":"2024-04-26T21:16:56.361962+0000","last_peered":"2024-04-26T21:16:56.361962+0000","last_clean":"2024-04-26T21:16:56.361962+0000","last_became_active":"2024-04-26T21:16:03.282315+0000","last_became_peered":"2024-04-26T21:16:03.282315+0000","last_unstale":"2024-04-26T21:16:56.361962+0000","last_undegraded":"2024-04-26T21:16:56.361962+0000","last_fullsized":"2024-04-26T21:16:56.361962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:13.358002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362813+0000","last_change":"2024-04-26T21:16:03.711992+0000","last_active":"2024-04-26T21:16:56.362813+0000","last_peered":"2024-04-26T21:16:56.362813+0000","last_clean":"2024-04-26T21:16:56.362813+0000","last_became_active":"2024-04-26T21:16:03.711755+0000","last_became_peered":"2024-04-26T21:16:03.711755+0000","last_unstale":"2024-04-26T21:16:56.362813+0000","last_undegraded":"2024-04-26T21:16:56.362813+0000","last_fullsized":"2024-04-26T21:16:56.362813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:01.321530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362836+0000","last_change":"2024-04-26T21:16:03.274722+0000","last_active":"2024-04-26T21:16:56.362836+0000","last_peered":"2024-04-26T21:16:56.362836+0000","last_clean":"2024-04-26T21:16:56.362836+0000","last_became_active":"2024-04-26T21:16:03.274399+0000","last_became_peered":"2024-04-26T21:16:03.274399+0000","last_unstale":"2024-04-26T21:16:56.362836+0000","last_undegraded":"2024-04-26T21:16:56.362836+0000","last_fullsized":"2024-04-26T21:16:56.362836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:35.247456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359442+0000","last_change":"2024-04-26T21:16:03.698344+0000","last_active":"2024-04-26T21:16:56.359442+0000","last_peered":"2024-04-26T21:16:56.359442+0000","last_clean":"2024-04-26T21:16:56.359442+0000","last_became_active":"2024-04-26T21:16:03.698037+0000","last_became_peered":"2024-04-26T21:16:03.698037+0000","last_unstale":"2024-04-26T21:16:56.359442+0000","last_undegraded":"2024-04-26T21:16:56.359442+0000","last_fullsized":"2024-04-26T21:16:56.359442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.573421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362176+0000","last_change":"2024-04-26T21:16:03.276459+0000","last_active":"2024-04-26T21:16:56.362176+0000","last_peered":"2024-04-26T21:16:56.362176+0000","last_clean":"2024-04-26T21:16:56.362176+0000","last_became_active":"2024-04-26T21:16:03.276108+0000","last_became_peered":"2024-04-26T21:16:03.276108+0000","last_unstale":"2024-04-26T21:16:56.362176+0000","last_undegraded":"2024-04-26T21:16:56.362176+0000","last_fullsized":"2024-04-26T21:16:56.362176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.364925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712741+0000","last_change":"2024-04-26T21:16:03.696030+0000","last_active":"2024-04-26T21:16:56.712741+0000","last_peered":"2024-04-26T21:16:56.712741+0000","last_clean":"2024-04-26T21:16:56.712741+0000","last_became_active":"2024-04-26T21:16:03.695717+0000","last_became_peered":"2024-04-26T21:16:03.695717+0000","last_unstale":"2024-04-26T21:16:56.712741+0000","last_undegraded":"2024-04-26T21:16:56.712741+0000","last_fullsized":"2024-04-26T21:16:56.712741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:55.702216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361784+0000","last_change":"2024-04-26T21:16:03.269188+0000","last_active":"2024-04-26T21:16:56.361784+0000","last_peered":"2024-04-26T21:16:56.361784+0000","last_clean":"2024-04-26T21:16:56.361784+0000","last_became_active":"2024-04-26T21:16:03.268850+0000","last_became_peered":"2024-04-26T21:16:03.268850+0000","last_unstale":"2024-04-26T21:16:56.361784+0000","last_undegraded":"2024-04-26T21:16:56.361784+0000","last_fullsized":"2024-04-26T21:16:56.361784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:54.183612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713804+0000","last_change":"2024-04-26T21:16:03.700284+0000","last_active":"2024-04-26T21:16:56.713804+0000","last_peered":"2024-04-26T21:16:56.713804+0000","last_clean":"2024-04-26T21:16:56.713804+0000","last_became_active":"2024-04-26T21:16:03.699965+0000","last_became_peered":"2024-04-26T21:16:03.699965+0000","last_unstale":"2024-04-26T21:16:56.713804+0000","last_undegraded":"2024-04-26T21:16:56.713804+0000","last_fullsized":"2024-04-26T21:16:56.713804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:40.547296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362424+0000","last_change":"2024-04-26T21:16:03.701370+0000","last_active":"2024-04-26T21:16:56.362424+0000","last_peered":"2024-04-26T21:16:56.362424+0000","last_clean":"2024-04-26T21:16:56.362424+0000","last_became_active":"2024-04-26T21:16:03.700585+0000","last_became_peered":"2024-04-26T21:16:03.700585+0000","last_unstale":"2024-04-26T21:16:56.362424+0000","last_undegraded":"2024-04-26T21:16:56.362424+0000","last_fullsized":"2024-04-26T21:16:56.362424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.348009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360613+0000","last_change":"2024-04-26T21:16:03.281141+0000","last_active":"2024-04-26T21:16:56.360613+0000","last_peered":"2024-04-26T21:16:56.360613+0000","last_clean":"2024-04-26T21:16:56.360613+0000","last_became_active":"2024-04-26T21:16:03.280812+0000","last_became_peered":"2024-04-26T21:16:03.280812+0000","last_unstale":"2024-04-26T21:16:56.360613+0000","last_undegraded":"2024-04-26T21:16:56.360613+0000","last_fullsized":"2024-04-26T21:16:56.360613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:11.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362156+0000","last_change":"2024-04-26T21:16:03.711312+0000","last_active":"2024-04-26T21:16:56.362156+0000","last_peered":"2024-04-26T21:16:56.362156+0000","last_clean":"2024-04-26T21:16:56.362156+0000","last_became_active":"2024-04-26T21:16:03.711026+0000","last_became_peered":"2024-04-26T21:16:03.711026+0000","last_unstale":"2024-04-26T21:16:56.362156+0000","last_undegraded":"2024-04-26T21:16:56.362156+0000","last_fullsized":"2024-04-26T21:16:56.362156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:10.185877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359923+0000","last_change":"2024-04-26T21:16:03.273428+0000","last_active":"2024-04-26T21:16:56.359923+0000","last_peered":"2024-04-26T21:16:56.359923+0000","last_clean":"2024-04-26T21:16:56.359923+0000","last_became_active":"2024-04-26T21:16:03.272854+0000","last_became_peered":"2024-04-26T21:16:03.272854+0000","last_unstale":"2024-04-26T21:16:56.359923+0000","last_undegraded":"2024-04-26T21:16:56.359923+0000","last_fullsized":"2024-04-26T21:16:56.359923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:48.735059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361618+0000","last_change":"2024-04-26T21:14:50.729892+0000","last_active":"2024-04-26T21:16:56.361618+0000","last_peered":"2024-04-26T21:16:56.361618+0000","last_clean":"2024-04-26T21:16:56.361618+0000","last_became_active":"2024-04-26T21:14:50.729430+0000","last_became_peered":"2024-04-26T21:14:50.729430+0000","last_unstale":"2024-04-26T21:16:56.361618+0000","last_undegraded":"2024-04-26T21:16:56.361618+0000","last_fullsized":"2024-04-26T21:16:56.361618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_clean_scrub_stamp":"2024-04-26T21:11:22.039289+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:13.510971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360509+0000","last_change":"2024-04-26T21:16:03.700702+0000","last_active":"2024-04-26T21:16:56.360509+0000","last_peered":"2024-04-26T21:16:56.360509+0000","last_clean":"2024-04-26T21:16:56.360509+0000","last_became_active":"2024-04-26T21:16:03.700412+0000","last_became_peered":"2024-04-26T21:16:03.700412+0000","last_unstale":"2024-04-26T21:16:56.360509+0000","last_undegraded":"2024-04-26T21:16:56.360509+0000","last_fullsized":"2024-04-26T21:16:56.360509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:49.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361824+0000","last_change":"2024-04-26T21:16:03.271730+0000","last_active":"2024-04-26T21:16:56.361824+0000","last_peered":"2024-04-26T21:16:56.361824+0000","last_clean":"2024-04-26T21:16:56.361824+0000","last_became_active":"2024-04-26T21:16:03.271401+0000","last_became_peered":"2024-04-26T21:16:03.271401+0000","last_unstale":"2024-04-26T21:16:56.361824+0000","last_undegraded":"2024-04-26T21:16:56.361824+0000","last_fullsized":"2024-04-26T21:16:56.361824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:16.469022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362223+0000","last_change":"2024-04-26T21:16:03.276229+0000","last_active":"2024-04-26T21:16:56.362223+0000","last_peered":"2024-04-26T21:16:56.362223+0000","last_clean":"2024-04-26T21:16:56.362223+0000","last_became_active":"2024-04-26T21:16:03.275910+0000","last_became_peered":"2024-04-26T21:16:03.275910+0000","last_unstale":"2024-04-26T21:16:56.362223+0000","last_undegraded":"2024-04-26T21:16:56.362223+0000","last_fullsized":"2024-04-26T21:16:56.362223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:03.932254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361647+0000","last_change":"2024-04-26T21:16:03.698293+0000","last_active":"2024-04-26T21:16:56.361647+0000","last_peered":"2024-04-26T21:16:56.361647+0000","last_clean":"2024-04-26T21:16:56.361647+0000","last_became_active":"2024-04-26T21:16:03.697935+0000","last_became_peered":"2024-04-26T21:16:03.697935+0000","last_unstale":"2024-04-26T21:16:56.361647+0000","last_undegraded":"2024-04-26T21:16:56.361647+0000","last_fullsized":"2024-04-26T21:16:56.361647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:54.292921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362587+0000","last_change":"2024-04-26T21:16:03.275162+0000","last_active":"2024-04-26T21:16:56.362587+0000","last_peered":"2024-04-26T21:16:56.362587+0000","last_clean":"2024-04-26T21:16:56.362587+0000","last_became_active":"2024-04-26T21:16:03.274791+0000","last_became_peered":"2024-04-26T21:16:03.274791+0000","last_unstale":"2024-04-26T21:16:56.362587+0000","last_undegraded":"2024-04-26T21:16:56.362587+0000","last_fullsized":"2024-04-26T21:16:56.362587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:15.583747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364592+0000","last_change":"2024-04-26T21:16:03.701473+0000","last_active":"2024-04-26T21:16:56.364592+0000","last_peered":"2024-04-26T21:16:56.364592+0000","last_clean":"2024-04-26T21:16:56.364592+0000","last_became_active":"2024-04-26T21:16:03.701070+0000","last_became_peered":"2024-04-26T21:16:03.701070+0000","last_unstale":"2024-04-26T21:16:56.364592+0000","last_undegraded":"2024-04-26T21:16:56.364592+0000","last_fullsized":"2024-04-26T21:16:56.364592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:25.373301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360117+0000","last_change":"2024-04-26T21:16:03.271036+0000","last_active":"2024-04-26T21:16:56.360117+0000","last_peered":"2024-04-26T21:16:56.360117+0000","last_clean":"2024-04-26T21:16:56.360117+0000","last_became_active":"2024-04-26T21:16:03.270686+0000","last_became_peered":"2024-04-26T21:16:03.270686+0000","last_unstale":"2024-04-26T21:16:56.360117+0000","last_undegraded":"2024-04-26T21:16:56.360117+0000","last_fullsized":"2024-04-26T21:16:56.360117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:21.025757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368114+0000","last_change":"2024-04-26T21:16:03.705314+0000","last_active":"2024-04-26T21:16:56.368114+0000","last_peered":"2024-04-26T21:16:56.368114+0000","last_clean":"2024-04-26T21:16:56.368114+0000","last_became_active":"2024-04-26T21:16:03.704952+0000","last_became_peered":"2024-04-26T21:16:03.704952+0000","last_unstale":"2024-04-26T21:16:56.368114+0000","last_undegraded":"2024-04-26T21:16:56.368114+0000","last_fullsized":"2024-04-26T21:16:56.368114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:57.363909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713329+0000","last_change":"2024-04-26T21:16:03.280995+0000","last_active":"2024-04-26T21:16:56.713329+0000","last_peered":"2024-04-26T21:16:56.713329+0000","last_clean":"2024-04-26T21:16:56.713329+0000","last_became_active":"2024-04-26T21:16:03.280615+0000","last_became_peered":"2024-04-26T21:16:03.280615+0000","last_unstale":"2024-04-26T21:16:56.713329+0000","last_undegraded":"2024-04-26T21:16:56.713329+0000","last_fullsized":"2024-04-26T21:16:56.713329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:32.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362097+0000","last_change":"2024-04-26T21:16:03.701218+0000","last_active":"2024-04-26T21:16:56.362097+0000","last_peered":"2024-04-26T21:16:56.362097+0000","last_clean":"2024-04-26T21:16:56.362097+0000","last_became_active":"2024-04-26T21:16:03.700895+0000","last_became_peered":"2024-04-26T21:16:03.700895+0000","last_unstale":"2024-04-26T21:16:56.362097+0000","last_undegraded":"2024-04-26T21:16:56.362097+0000","last_fullsized":"2024-04-26T21:16:56.362097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:44.104683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362857+0000","last_change":"2024-04-26T21:16:03.277235+0000","last_active":"2024-04-26T21:16:56.362857+0000","last_peered":"2024-04-26T21:16:56.362857+0000","last_clean":"2024-04-26T21:16:56.362857+0000","last_became_active":"2024-04-26T21:16:03.276901+0000","last_became_peered":"2024-04-26T21:16:03.276901+0000","last_unstale":"2024-04-26T21:16:56.362857+0000","last_undegraded":"2024-04-26T21:16:56.362857+0000","last_fullsized":"2024-04-26T21:16:56.362857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:16.297883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359135+0000","last_change":"2024-04-26T21:16:03.704318+0000","last_active":"2024-04-26T21:16:56.359135+0000","last_peered":"2024-04-26T21:16:56.359135+0000","last_clean":"2024-04-26T21:16:56.359135+0000","last_became_active":"2024-04-26T21:16:03.703907+0000","last_became_peered":"2024-04-26T21:16:03.703907+0000","last_unstale":"2024-04-26T21:16:56.359135+0000","last_undegraded":"2024-04-26T21:16:56.359135+0000","last_fullsized":"2024-04-26T21:16:56.359135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:16.431246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372105+0000","last_change":"2024-04-26T21:16:56.371135+0000","last_active":"2024-04-26T21:16:56.372105+0000","last_peered":"2024-04-26T21:16:56.372105+0000","last_clean":"2024-04-26T21:16:56.372105+0000","last_became_active":"2024-04-26T21:16:56.370666+0000","last_became_peered":"2024-04-26T21:16:56.370666+0000","last_unstale":"2024-04-26T21:16:56.372105+0000","last_undegraded":"2024-04-26T21:16:56.372105+0000","last_fullsized":"2024-04-26T21:16:56.372105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:05.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362114+0000","last_change":"2024-04-26T21:16:03.705026+0000","last_active":"2024-04-26T21:16:56.362114+0000","last_peered":"2024-04-26T21:16:56.362114+0000","last_clean":"2024-04-26T21:16:56.362114+0000","last_became_active":"2024-04-26T21:16:03.703336+0000","last_became_peered":"2024-04-26T21:16:03.703336+0000","last_unstale":"2024-04-26T21:16:56.362114+0000","last_undegraded":"2024-04-26T21:16:56.362114+0000","last_fullsized":"2024-04-26T21:16:56.362114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:11.746362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362335+0000","last_change":"2024-04-26T21:16:03.279263+0000","last_active":"2024-04-26T21:16:56.362335+0000","last_peered":"2024-04-26T21:16:56.362335+0000","last_clean":"2024-04-26T21:16:56.362335+0000","last_became_active":"2024-04-26T21:16:03.278948+0000","last_became_peered":"2024-04-26T21:16:03.278948+0000","last_unstale":"2024-04-26T21:16:56.362335+0000","last_undegraded":"2024-04-26T21:16:56.362335+0000","last_fullsized":"2024-04-26T21:16:56.362335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:39.078346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362036+0000","last_change":"2024-04-26T21:16:03.702913+0000","last_active":"2024-04-26T21:16:56.362036+0000","last_peered":"2024-04-26T21:16:56.362036+0000","last_clean":"2024-04-26T21:16:56.362036+0000","last_became_active":"2024-04-26T21:16:03.702507+0000","last_became_peered":"2024-04-26T21:16:03.702507+0000","last_unstale":"2024-04-26T21:16:56.362036+0000","last_undegraded":"2024-04-26T21:16:56.362036+0000","last_fullsized":"2024-04-26T21:16:56.362036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:51.846983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364220+0000","last_change":"2024-04-26T21:16:03.281898+0000","last_active":"2024-04-26T21:16:56.364220+0000","last_peered":"2024-04-26T21:16:56.364220+0000","last_clean":"2024-04-26T21:16:56.364220+0000","last_became_active":"2024-04-26T21:16:03.281544+0000","last_became_peered":"2024-04-26T21:16:03.281544+0000","last_unstale":"2024-04-26T21:16:56.364220+0000","last_undegraded":"2024-04-26T21:16:56.364220+0000","last_fullsized":"2024-04-26T21:16:56.364220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:39.061050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362089+0000","last_change":"2024-04-26T21:16:03.703006+0000","last_active":"2024-04-26T21:16:56.362089+0000","last_peered":"2024-04-26T21:16:56.362089+0000","last_clean":"2024-04-26T21:16:56.362089+0000","last_became_active":"2024-04-26T21:16:03.702705+0000","last_became_peered":"2024-04-26T21:16:03.702705+0000","last_unstale":"2024-04-26T21:16:56.362089+0000","last_undegraded":"2024-04-26T21:16:56.362089+0000","last_fullsized":"2024-04-26T21:16:56.362089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:11.885993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364037+0000","last_change":"2024-04-26T21:16:03.280891+0000","last_active":"2024-04-26T21:16:56.364037+0000","last_peered":"2024-04-26T21:16:56.364037+0000","last_clean":"2024-04-26T21:16:56.364037+0000","last_became_active":"2024-04-26T21:16:03.280577+0000","last_became_peered":"2024-04-26T21:16:03.280577+0000","last_unstale":"2024-04-26T21:16:56.364037+0000","last_undegraded":"2024-04-26T21:16:56.364037+0000","last_fullsized":"2024-04-26T21:16:56.364037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:28.698997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.378258+0000","last_change":"2024-04-26T21:16:56.378258+0000","last_active":"2024-04-26T21:16:56.378258+0000","last_peered":"2024-04-26T21:16:56.378258+0000","last_clean":"2024-04-26T21:16:56.378258+0000","last_became_active":"2024-04-26T21:16:56.371874+0000","last_became_peered":"2024-04-26T21:16:56.371874+0000","last_unstale":"2024-04-26T21:16:56.378258+0000","last_undegraded":"2024-04-26T21:16:56.378258+0000","last_fullsized":"2024-04-26T21:16:56.378258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:56.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360941+0000","last_change":"2024-04-26T21:16:03.276651+0000","last_active":"2024-04-26T21:16:56.360941+0000","last_peered":"2024-04-26T21:16:56.360941+0000","last_clean":"2024-04-26T21:16:56.360941+0000","last_became_active":"2024-04-26T21:16:03.276347+0000","last_became_peered":"2024-04-26T21:16:03.276347+0000","last_unstale":"2024-04-26T21:16:56.360941+0000","last_undegraded":"2024-04-26T21:16:56.360941+0000","last_fullsized":"2024-04-26T21:16:56.360941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:48.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362100+0000","last_change":"2024-04-26T21:16:03.705647+0000","last_active":"2024-04-26T21:16:56.362100+0000","last_peered":"2024-04-26T21:16:56.362100+0000","last_clean":"2024-04-26T21:16:56.362100+0000","last_became_active":"2024-04-26T21:16:03.705251+0000","last_became_peered":"2024-04-26T21:16:03.705251+0000","last_unstale":"2024-04-26T21:16:56.362100+0000","last_undegraded":"2024-04-26T21:16:56.362100+0000","last_fullsized":"2024-04-26T21:16:56.362100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:17.427221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359241+0000","last_change":"2024-04-26T21:16:03.701759+0000","last_active":"2024-04-26T21:16:56.359241+0000","last_peered":"2024-04-26T21:16:56.359241+0000","last_clean":"2024-04-26T21:16:56.359241+0000","last_became_active":"2024-04-26T21:16:03.701382+0000","last_became_peered":"2024-04-26T21:16:03.701382+0000","last_unstale":"2024-04-26T21:16:56.359241+0000","last_undegraded":"2024-04-26T21:16:56.359241+0000","last_fullsized":"2024-04-26T21:16:56.359241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.789455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372345+0000","last_change":"2024-04-26T21:16:56.372345+0000","last_active":"2024-04-26T21:16:56.372345+0000","last_peered":"2024-04-26T21:16:56.372345+0000","last_clean":"2024-04-26T21:16:56.372345+0000","last_became_active":"2024-04-26T21:16:56.371900+0000","last_became_peered":"2024-04-26T21:16:56.371900+0000","last_unstale":"2024-04-26T21:16:56.372345+0000","last_undegraded":"2024-04-26T21:16:56.372345+0000","last_fullsized":"2024-04-26T21:16:56.372345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:42.192142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361562+0000","last_change":"2024-04-26T21:16:03.702012+0000","last_active":"2024-04-26T21:16:56.361562+0000","last_peered":"2024-04-26T21:16:56.361562+0000","last_clean":"2024-04-26T21:16:56.361562+0000","last_became_active":"2024-04-26T21:16:03.701066+0000","last_became_peered":"2024-04-26T21:16:03.701066+0000","last_unstale":"2024-04-26T21:16:56.361562+0000","last_undegraded":"2024-04-26T21:16:56.361562+0000","last_fullsized":"2024-04-26T21:16:56.361562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:45.337320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367378+0000","last_change":"2024-04-26T21:16:03.702640+0000","last_active":"2024-04-26T21:16:56.367378+0000","last_peered":"2024-04-26T21:16:56.367378+0000","last_clean":"2024-04-26T21:16:56.367378+0000","last_became_active":"2024-04-26T21:16:03.702339+0000","last_became_peered":"2024-04-26T21:16:03.702339+0000","last_unstale":"2024-04-26T21:16:56.367378+0000","last_undegraded":"2024-04-26T21:16:56.367378+0000","last_fullsized":"2024-04-26T21:16:56.367378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:24.083734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361731+0000","last_change":"2024-04-26T21:16:03.708369+0000","last_active":"2024-04-26T21:16:56.361731+0000","last_peered":"2024-04-26T21:16:56.361731+0000","last_clean":"2024-04-26T21:16:56.361731+0000","last_became_active":"2024-04-26T21:16:03.708050+0000","last_became_peered":"2024-04-26T21:16:03.708050+0000","last_unstale":"2024-04-26T21:16:56.361731+0000","last_undegraded":"2024-04-26T21:16:56.361731+0000","last_fullsized":"2024-04-26T21:16:56.361731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:58.200043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359681+0000","last_change":"2024-04-26T21:16:03.700362+0000","last_active":"2024-04-26T21:16:56.359681+0000","last_peered":"2024-04-26T21:16:56.359681+0000","last_clean":"2024-04-26T21:16:56.359681+0000","last_became_active":"2024-04-26T21:16:03.700053+0000","last_became_peered":"2024-04-26T21:16:03.700053+0000","last_unstale":"2024-04-26T21:16:56.359681+0000","last_undegraded":"2024-04-26T21:16:56.359681+0000","last_fullsized":"2024-04-26T21:16:56.359681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:23.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363986+0000","last_change":"2024-04-26T21:16:03.707717+0000","last_active":"2024-04-26T21:16:56.363986+0000","last_peered":"2024-04-26T21:16:56.363986+0000","last_clean":"2024-04-26T21:16:56.363986+0000","last_became_active":"2024-04-26T21:16:03.707343+0000","last_became_peered":"2024-04-26T21:16:03.707343+0000","last_unstale":"2024-04-26T21:16:56.363986+0000","last_undegraded":"2024-04-26T21:16:56.363986+0000","last_fullsized":"2024-04-26T21:16:56.363986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:46.261231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369892+0000","last_change":"2024-04-26T21:16:56.369892+0000","last_active":"2024-04-26T21:16:56.369892+0000","last_peered":"2024-04-26T21:16:56.369892+0000","last_clean":"2024-04-26T21:16:56.369892+0000","last_became_active":"2024-04-26T21:16:56.369416+0000","last_became_peered":"2024-04-26T21:16:56.369416+0000","last_unstale":"2024-04-26T21:16:56.369892+0000","last_undegraded":"2024-04-26T21:16:56.369892+0000","last_fullsized":"2024-04-26T21:16:56.369892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:45.708969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362186+0000","last_change":"2024-04-26T21:16:03.699537+0000","last_active":"2024-04-26T21:16:56.362186+0000","last_peered":"2024-04-26T21:16:56.362186+0000","last_clean":"2024-04-26T21:16:56.362186+0000","last_became_active":"2024-04-26T21:16:03.699186+0000","last_became_peered":"2024-04-26T21:16:03.699186+0000","last_unstale":"2024-04-26T21:16:56.362186+0000","last_undegraded":"2024-04-26T21:16:56.362186+0000","last_fullsized":"2024-04-26T21:16:56.362186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:34.784929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362274+0000","last_change":"2024-04-26T21:16:03.709940+0000","last_active":"2024-04-26T21:16:56.362274+0000","last_peered":"2024-04-26T21:16:56.362274+0000","last_clean":"2024-04-26T21:16:56.362274+0000","last_became_active":"2024-04-26T21:16:03.707255+0000","last_became_peered":"2024-04-26T21:16:03.707255+0000","last_unstale":"2024-04-26T21:16:56.362274+0000","last_undegraded":"2024-04-26T21:16:56.362274+0000","last_fullsized":"2024-04-26T21:16:56.362274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:57.652194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713080+0000","last_change":"2024-04-26T21:16:03.711139+0000","last_active":"2024-04-26T21:16:56.713080+0000","last_peered":"2024-04-26T21:16:56.713080+0000","last_clean":"2024-04-26T21:16:56.713080+0000","last_became_active":"2024-04-26T21:16:03.709720+0000","last_became_peered":"2024-04-26T21:16:03.709720+0000","last_unstale":"2024-04-26T21:16:56.713080+0000","last_undegraded":"2024-04-26T21:16:56.713080+0000","last_fullsized":"2024-04-26T21:16:56.713080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:10.021914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361384+0000","last_change":"2024-04-26T21:16:03.699140+0000","last_active":"2024-04-26T21:16:56.361384+0000","last_peered":"2024-04-26T21:16:56.361384+0000","last_clean":"2024-04-26T21:16:56.361384+0000","last_became_active":"2024-04-26T21:16:03.698824+0000","last_became_peered":"2024-04-26T21:16:03.698824+0000","last_unstale":"2024-04-26T21:16:56.361384+0000","last_undegraded":"2024-04-26T21:16:56.361384+0000","last_fullsized":"2024-04-26T21:16:56.361384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:55.896278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359230+0000","last_change":"2024-04-26T21:16:03.708258+0000","last_active":"2024-04-26T21:16:56.359230+0000","last_peered":"2024-04-26T21:16:56.359230+0000","last_clean":"2024-04-26T21:16:56.359230+0000","last_became_active":"2024-04-26T21:16:03.707932+0000","last_became_peered":"2024-04-26T21:16:03.707932+0000","last_unstale":"2024-04-26T21:16:56.359230+0000","last_undegraded":"2024-04-26T21:16:56.359230+0000","last_fullsized":"2024-04-26T21:16:56.359230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:56.758307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244196,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53330,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":163,"num_read_kb":135,"num_write":74,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":159990,"data_compressed":5568,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":458793,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":1102004,"data_compressed":43255,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459492,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459397,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1049860,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":458910,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":511325,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":458883,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":511359,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":459014,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":458815,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1049460,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":52532,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52566,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":52726,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:17:24.893 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:17:25.135 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:25.519 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:25 smithi129 ceph-mon[29237]: pgmap v333: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:25.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:25 smithi171 ceph-mon[33478]: pgmap v333: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:25.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:25 smithi150 ceph-mon[32290]: pgmap v333: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:26.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.888+0000 7f29c3530700 1 -- 172.21.15.129:0/1374415078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06c510 msgr2=0x7f29bc110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.888+0000 7f29c3530700 1 --2- 172.21.15.129:0/1374415078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06c510 0x7f29bc110310 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f29b400b0a0 tx=0x7f29b400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.888+0000 7f29c3530700 1 -- 172.21.15.129:0/1374415078 shutdown_connections 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.888+0000 7f29c3530700 1 --2- 172.21.15.129:0/1374415078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06c510 0x7f29bc110310 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.888+0000 7f29c3530700 1 --2- 172.21.15.129:0/1374415078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc109610 0x7f29bc1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.888+0000 7f29c3530700 1 --2- 172.21.15.129:0/1374415078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc06bb70 0x7f29bc06bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.888+0000 7f29c3530700 1 -- 172.21.15.129:0/1374415078 >> 172.21.15.129:0/1374415078 conn(0x7f29bc079210 msgr2=0x7f29bc079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.888+0000 7f29c3530700 1 -- 172.21.15.129:0/1374415078 shutdown_connections 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.888+0000 7f29c3530700 1 -- 172.21.15.129:0/1374415078 wait complete. 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c3530700 1 Processor -- start 2024-04-26T21:17:26.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c3530700 1 -- start start 2024-04-26T21:17:26.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c3530700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06bb70 0x7f29bc1026d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:26.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c3530700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc06c510 0x7f29bc102c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:26.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c3530700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc109610 0x7f29bc103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:26.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c3530700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc113680 con 0x7f29bc06bb70 2024-04-26T21:17:26.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c3530700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc113500 con 0x7f29bc06c510 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c3530700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc113800 con 0x7f29bc109610 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c12cc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06bb70 0x7f29bc1026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c12cc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06bb70 0x7f29bc1026d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48670/0 (socket says 172.21.15.129:48670) 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c12cc700 1 -- 172.21.15.129:0/1090539209 learned_addr learned my addr 172.21.15.129:0/1090539209 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c0acb700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc06c510 0x7f29bc102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c1acd700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc109610 0x7f29bc103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c12cc700 1 -- 172.21.15.129:0/1090539209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc109610 msgr2=0x7f29bc103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c12cc700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc109610 0x7f29bc103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.889+0000 7f29c12cc700 1 -- 172.21.15.129:0/1090539209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc06c510 msgr2=0x7f29bc102c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.890+0000 7f29c12cc700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc06c510 0x7f29bc102c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.890+0000 7f29c12cc700 1 -- 172.21.15.129:0/1090539209 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc1a7ed0 con 0x7f29bc06bb70 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.890+0000 7f29c0acb700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc06c510 0x7f29bc102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.890+0000 7f29c12cc700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06bb70 0x7f29bc1026d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c9b0 tx=0x7f29b800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:26.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.890+0000 7f29b27fc700 1 -- 172.21.15.129:0/1090539209 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8007b00 con 0x7f29bc06bb70 2024-04-26T21:17:26.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.890+0000 7f29b27fc700 1 -- 172.21.15.129:0/1090539209 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b800ce50 con 0x7f29bc06bb70 2024-04-26T21:17:26.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.890+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc1a81c0 con 0x7f29bc06bb70 2024-04-26T21:17:26.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.890+0000 7f29b27fc700 1 -- 172.21.15.129:0/1090539209 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8018660 con 0x7f29bc06bb70 2024-04-26T21:17:26.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.890+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1a8700 con 0x7f29bc06bb70 2024-04-26T21:17:26.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.891+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc104b80 con 0x7f29bc06bb70 2024-04-26T21:17:26.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.892+0000 7f29b27fc700 1 -- 172.21.15.129:0/1090539209 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f29b800f450 con 0x7f29bc06bb70 2024-04-26T21:17:26.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.893+0000 7f29b27fc700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f29a80a3cd0 0x7f29a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:26.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.893+0000 7f29c0acb700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f29a80a3cd0 0x7f29a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:26.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.893+0000 7f29b27fc700 1 -- 172.21.15.129:0/1090539209 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f29b80d0530 con 0x7f29bc06bb70 2024-04-26T21:17:26.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.893+0000 7f29c0acb700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f29a80a3cd0 0x7f29a80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29bc079710 tx=0x7f29ac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:26.897+0000 7f29b27fc700 1 -- 172.21.15.129:0/1090539209 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f29b8014030 con 0x7f29bc06bb70 2024-04-26T21:17:27.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.093+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29bc101700 con 0x7f29a80a3cd0 2024-04-26T21:17:27.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.100+0000 7f29b27fc700 1 -- 172.21.15.129:0/1090539209 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7f29bc101700 con 0x7f29a80a3cd0 2024-04-26T21:17:27.104 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:27.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f29a80a3cd0 msgr2=0x7f29a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:27.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f29a80a3cd0 0x7f29a80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29bc079710 tx=0x7f29ac009190 comp rx=0 tx=0).stop 2024-04-26T21:17:27.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06bb70 msgr2=0x7f29bc1026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:27.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06bb70 0x7f29bc1026d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c9b0 tx=0x7f29b800ccc0 comp rx=0 tx=0).stop 2024-04-26T21:17:27.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 shutdown_connections 2024-04-26T21:17:27.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f29a80a3cd0 0x7f29a80a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc109610 0x7f29bc103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29bc06bb70 0x7f29bc1026d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 --2- 172.21.15.129:0/1090539209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc06c510 0x7f29bc102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 >> 172.21.15.129:0/1090539209 conn(0x7f29bc079210 msgr2=0x7f29bc100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 shutdown_connections 2024-04-26T21:17:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:27.103+0000 7f29c3530700 1 -- 172.21.15.129:0/1090539209 wait complete. 2024-04-26T21:17:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-26T21:17:27.894 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-04-26T21:17:25.297693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643932,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":289,"num_read_kb":244,"num_write":257,"num_write_kb":3014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360704,"kb_used_data":11484,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581120,"statfs":{"total":1151940427776,"available":1151571066880,"internally_reserved":0,"allocated":11759616,"data_stored":7437312,"data_compressed":182064,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003380"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359578+0000","last_change":"2024-04-26T21:16:03.700581+0000","last_active":"2024-04-26T21:16:56.359578+0000","last_peered":"2024-04-26T21:16:56.359578+0000","last_clean":"2024-04-26T21:16:56.359578+0000","last_became_active":"2024-04-26T21:16:03.700280+0000","last_became_peered":"2024-04-26T21:16:03.700280+0000","last_unstale":"2024-04-26T21:16:56.359578+0000","last_undegraded":"2024-04-26T21:16:56.359578+0000","last_fullsized":"2024-04-26T21:16:56.359578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:01.329784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368025+0000","last_change":"2024-04-26T21:16:03.702058+0000","last_active":"2024-04-26T21:16:56.368025+0000","last_peered":"2024-04-26T21:16:56.368025+0000","last_clean":"2024-04-26T21:16:56.368025+0000","last_became_active":"2024-04-26T21:16:03.701728+0000","last_became_peered":"2024-04-26T21:16:03.701728+0000","last_unstale":"2024-04-26T21:16:56.368025+0000","last_undegraded":"2024-04-26T21:16:56.368025+0000","last_fullsized":"2024-04-26T21:16:56.368025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:12.718225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367712+0000","last_change":"2024-04-26T21:16:03.710461+0000","last_active":"2024-04-26T21:16:56.367712+0000","last_peered":"2024-04-26T21:16:56.367712+0000","last_clean":"2024-04-26T21:16:56.367712+0000","last_became_active":"2024-04-26T21:16:03.710130+0000","last_became_peered":"2024-04-26T21:16:03.710130+0000","last_unstale":"2024-04-26T21:16:56.367712+0000","last_undegraded":"2024-04-26T21:16:56.367712+0000","last_fullsized":"2024-04-26T21:16:56.367712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:21.294677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360172+0000","last_change":"2024-04-26T21:16:03.706836+0000","last_active":"2024-04-26T21:16:56.360172+0000","last_peered":"2024-04-26T21:16:56.360172+0000","last_clean":"2024-04-26T21:16:56.360172+0000","last_became_active":"2024-04-26T21:16:03.706515+0000","last_became_peered":"2024-04-26T21:16:03.706515+0000","last_unstale":"2024-04-26T21:16:56.360172+0000","last_undegraded":"2024-04-26T21:16:56.360172+0000","last_fullsized":"2024-04-26T21:16:56.360172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:56.198337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362204+0000","last_change":"2024-04-26T21:16:03.704865+0000","last_active":"2024-04-26T21:16:56.362204+0000","last_peered":"2024-04-26T21:16:56.362204+0000","last_clean":"2024-04-26T21:16:56.362204+0000","last_became_active":"2024-04-26T21:16:03.704560+0000","last_became_peered":"2024-04-26T21:16:03.704560+0000","last_unstale":"2024-04-26T21:16:56.362204+0000","last_undegraded":"2024-04-26T21:16:56.362204+0000","last_fullsized":"2024-04-26T21:16:56.362204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:51:03.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712989+0000","last_change":"2024-04-26T21:16:03.700265+0000","last_active":"2024-04-26T21:16:56.712989+0000","last_peered":"2024-04-26T21:16:56.712989+0000","last_clean":"2024-04-26T21:16:56.712989+0000","last_became_active":"2024-04-26T21:16:03.699878+0000","last_became_peered":"2024-04-26T21:16:03.699878+0000","last_unstale":"2024-04-26T21:16:56.712989+0000","last_undegraded":"2024-04-26T21:16:56.712989+0000","last_fullsized":"2024-04-26T21:16:56.712989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:12.907254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361631+0000","last_change":"2024-04-26T21:16:03.697569+0000","last_active":"2024-04-26T21:16:56.361631+0000","last_peered":"2024-04-26T21:16:56.361631+0000","last_clean":"2024-04-26T21:16:56.361631+0000","last_became_active":"2024-04-26T21:16:03.697262+0000","last_became_peered":"2024-04-26T21:16:03.697262+0000","last_unstale":"2024-04-26T21:16:56.361631+0000","last_undegraded":"2024-04-26T21:16:56.361631+0000","last_fullsized":"2024-04-26T21:16:56.361631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:28.645750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363218+0000","last_change":"2024-04-26T21:16:03.705788+0000","last_active":"2024-04-26T21:16:56.363218+0000","last_peered":"2024-04-26T21:16:56.363218+0000","last_clean":"2024-04-26T21:16:56.363218+0000","last_became_active":"2024-04-26T21:16:03.705479+0000","last_became_peered":"2024-04-26T21:16:03.705479+0000","last_unstale":"2024-04-26T21:16:56.363218+0000","last_undegraded":"2024-04-26T21:16:56.363218+0000","last_fullsized":"2024-04-26T21:16:56.363218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:27.306777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713655+0000","last_change":"2024-04-26T21:16:03.709716+0000","last_active":"2024-04-26T21:16:56.713655+0000","last_peered":"2024-04-26T21:16:56.713655+0000","last_clean":"2024-04-26T21:16:56.713655+0000","last_became_active":"2024-04-26T21:16:03.709219+0000","last_became_peered":"2024-04-26T21:16:03.709219+0000","last_unstale":"2024-04-26T21:16:56.713655+0000","last_undegraded":"2024-04-26T21:16:56.713655+0000","last_fullsized":"2024-04-26T21:16:56.713655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:18.936075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713179+0000","last_change":"2024-04-26T21:16:03.694745+0000","last_active":"2024-04-26T21:16:56.713179+0000","last_peered":"2024-04-26T21:16:56.713179+0000","last_clean":"2024-04-26T21:16:56.713179+0000","last_became_active":"2024-04-26T21:16:03.694044+0000","last_became_peered":"2024-04-26T21:16:03.694044+0000","last_unstale":"2024-04-26T21:16:56.713179+0000","last_undegraded":"2024-04-26T21:16:56.713179+0000","last_fullsized":"2024-04-26T21:16:56.713179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:13.060467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362033+0000","last_change":"2024-04-26T21:16:03.706434+0000","last_active":"2024-04-26T21:16:56.362033+0000","last_peered":"2024-04-26T21:16:56.362033+0000","last_clean":"2024-04-26T21:16:56.362033+0000","last_became_active":"2024-04-26T21:16:03.706093+0000","last_became_peered":"2024-04-26T21:16:03.706093+0000","last_unstale":"2024-04-26T21:16:56.362033+0000","last_undegraded":"2024-04-26T21:16:56.362033+0000","last_fullsized":"2024-04-26T21:16:56.362033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:04.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360084+0000","last_change":"2024-04-26T21:16:03.707672+0000","last_active":"2024-04-26T21:16:56.360084+0000","last_peered":"2024-04-26T21:16:56.360084+0000","last_clean":"2024-04-26T21:16:56.360084+0000","last_became_active":"2024-04-26T21:16:03.707344+0000","last_became_peered":"2024-04-26T21:16:03.707344+0000","last_unstale":"2024-04-26T21:16:56.360084+0000","last_undegraded":"2024-04-26T21:16:56.360084+0000","last_fullsized":"2024-04-26T21:16:56.360084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:07.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361281+0000","last_change":"2024-04-26T21:16:03.699816+0000","last_active":"2024-04-26T21:16:56.361281+0000","last_peered":"2024-04-26T21:16:56.361281+0000","last_clean":"2024-04-26T21:16:56.361281+0000","last_became_active":"2024-04-26T21:16:03.699514+0000","last_became_peered":"2024-04-26T21:16:03.699514+0000","last_unstale":"2024-04-26T21:16:56.361281+0000","last_undegraded":"2024-04-26T21:16:56.361281+0000","last_fullsized":"2024-04-26T21:16:56.361281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:59.230438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367152+0000","last_change":"2024-04-26T21:16:03.709627+0000","last_active":"2024-04-26T21:16:56.367152+0000","last_peered":"2024-04-26T21:16:56.367152+0000","last_clean":"2024-04-26T21:16:56.367152+0000","last_became_active":"2024-04-26T21:16:03.709238+0000","last_became_peered":"2024-04-26T21:16:03.709238+0000","last_unstale":"2024-04-26T21:16:56.367152+0000","last_undegraded":"2024-04-26T21:16:56.367152+0000","last_fullsized":"2024-04-26T21:16:56.367152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:55.303059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.366111+0000","last_change":"2024-04-26T21:16:03.702275+0000","last_active":"2024-04-26T21:16:56.366111+0000","last_peered":"2024-04-26T21:16:56.366111+0000","last_clean":"2024-04-26T21:16:56.366111+0000","last_became_active":"2024-04-26T21:16:03.701966+0000","last_became_peered":"2024-04-26T21:16:03.701966+0000","last_unstale":"2024-04-26T21:16:56.366111+0000","last_undegraded":"2024-04-26T21:16:56.366111+0000","last_fullsized":"2024-04-26T21:16:56.366111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:02.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713281+0000","last_change":"2024-04-26T21:16:03.711979+0000","last_active":"2024-04-26T21:16:56.713281+0000","last_peered":"2024-04-26T21:16:56.713281+0000","last_clean":"2024-04-26T21:16:56.713281+0000","last_became_active":"2024-04-26T21:16:03.711637+0000","last_became_peered":"2024-04-26T21:16:03.711637+0000","last_unstale":"2024-04-26T21:16:56.713281+0000","last_undegraded":"2024-04-26T21:16:56.713281+0000","last_fullsized":"2024-04-26T21:16:56.713281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:27.817993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359639+0000","last_change":"2024-04-26T21:16:03.701303+0000","last_active":"2024-04-26T21:16:56.359639+0000","last_peered":"2024-04-26T21:16:56.359639+0000","last_clean":"2024-04-26T21:16:56.359639+0000","last_became_active":"2024-04-26T21:16:03.701001+0000","last_became_peered":"2024-04-26T21:16:03.701001+0000","last_unstale":"2024-04-26T21:16:56.359639+0000","last_undegraded":"2024-04-26T21:16:56.359639+0000","last_fullsized":"2024-04-26T21:16:56.359639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:08.349915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362455+0000","last_change":"2024-04-26T21:16:03.696775+0000","last_active":"2024-04-26T21:16:56.362455+0000","last_peered":"2024-04-26T21:16:56.362455+0000","last_clean":"2024-04-26T21:16:56.362455+0000","last_became_active":"2024-04-26T21:16:03.696433+0000","last_became_peered":"2024-04-26T21:16:03.696433+0000","last_unstale":"2024-04-26T21:16:56.362455+0000","last_undegraded":"2024-04-26T21:16:56.362455+0000","last_fullsized":"2024-04-26T21:16:56.362455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:50.029169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361906+0000","last_change":"2024-04-26T21:16:03.698188+0000","last_active":"2024-04-26T21:16:56.361906+0000","last_peered":"2024-04-26T21:16:56.361906+0000","last_clean":"2024-04-26T21:16:56.361906+0000","last_became_active":"2024-04-26T21:16:03.697882+0000","last_became_peered":"2024-04-26T21:16:03.697882+0000","last_unstale":"2024-04-26T21:16:56.361906+0000","last_undegraded":"2024-04-26T21:16:56.361906+0000","last_fullsized":"2024-04-26T21:16:56.361906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:04.737619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362709+0000","last_change":"2024-04-26T21:16:03.275296+0000","last_active":"2024-04-26T21:16:56.362709+0000","last_peered":"2024-04-26T21:16:56.362709+0000","last_clean":"2024-04-26T21:16:56.362709+0000","last_became_active":"2024-04-26T21:16:03.274733+0000","last_became_peered":"2024-04-26T21:16:03.274733+0000","last_unstale":"2024-04-26T21:16:56.362709+0000","last_undegraded":"2024-04-26T21:16:56.362709+0000","last_fullsized":"2024-04-26T21:16:56.362709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:16.512934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362172+0000","last_change":"2024-04-26T21:16:03.705823+0000","last_active":"2024-04-26T21:16:56.362172+0000","last_peered":"2024-04-26T21:16:56.362172+0000","last_clean":"2024-04-26T21:16:56.362172+0000","last_became_active":"2024-04-26T21:16:03.705512+0000","last_became_peered":"2024-04-26T21:16:03.705512+0000","last_unstale":"2024-04-26T21:16:56.362172+0000","last_undegraded":"2024-04-26T21:16:56.362172+0000","last_fullsized":"2024-04-26T21:16:56.362172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:17.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364297+0000","last_change":"2024-04-26T21:16:03.285008+0000","last_active":"2024-04-26T21:16:56.364297+0000","last_peered":"2024-04-26T21:16:56.364297+0000","last_clean":"2024-04-26T21:16:56.364297+0000","last_became_active":"2024-04-26T21:16:03.284679+0000","last_became_peered":"2024-04-26T21:16:03.284679+0000","last_unstale":"2024-04-26T21:16:56.364297+0000","last_undegraded":"2024-04-26T21:16:56.364297+0000","last_fullsized":"2024-04-26T21:16:56.364297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:35.404086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368858+0000","last_change":"2024-04-26T21:16:03.710571+0000","last_active":"2024-04-26T21:16:56.368858+0000","last_peered":"2024-04-26T21:16:56.368858+0000","last_clean":"2024-04-26T21:16:56.368858+0000","last_became_active":"2024-04-26T21:16:03.710237+0000","last_became_peered":"2024-04-26T21:16:03.710237+0000","last_unstale":"2024-04-26T21:16:56.368858+0000","last_undegraded":"2024-04-26T21:16:56.368858+0000","last_fullsized":"2024-04-26T21:16:56.368858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:53.188554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369209+0000","last_change":"2024-04-26T21:16:56.368746+0000","last_active":"2024-04-26T21:16:56.369209+0000","last_peered":"2024-04-26T21:16:56.369209+0000","last_clean":"2024-04-26T21:16:56.369209+0000","last_became_active":"2024-04-26T21:16:56.365062+0000","last_became_peered":"2024-04-26T21:16:56.365062+0000","last_unstale":"2024-04-26T21:16:56.369209+0000","last_undegraded":"2024-04-26T21:16:56.369209+0000","last_fullsized":"2024-04-26T21:16:56.369209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:36.165146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713534+0000","last_change":"2024-04-26T21:16:03.711355+0000","last_active":"2024-04-26T21:16:56.713534+0000","last_peered":"2024-04-26T21:16:56.713534+0000","last_clean":"2024-04-26T21:16:56.713534+0000","last_became_active":"2024-04-26T21:16:03.710885+0000","last_became_peered":"2024-04-26T21:16:03.710885+0000","last_unstale":"2024-04-26T21:16:56.713534+0000","last_undegraded":"2024-04-26T21:16:56.713534+0000","last_fullsized":"2024-04-26T21:16:56.713534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:10.515590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362768+0000","last_change":"2024-04-26T21:16:03.278484+0000","last_active":"2024-04-26T21:16:56.362768+0000","last_peered":"2024-04-26T21:16:56.362768+0000","last_clean":"2024-04-26T21:16:56.362768+0000","last_became_active":"2024-04-26T21:16:03.277402+0000","last_became_peered":"2024-04-26T21:16:03.277402+0000","last_unstale":"2024-04-26T21:16:56.362768+0000","last_undegraded":"2024-04-26T21:16:56.362768+0000","last_fullsized":"2024-04-26T21:16:56.362768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:21.807942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714671+0000","last_change":"2024-04-26T21:16:56.714671+0000","last_active":"2024-04-26T21:16:56.714671+0000","last_peered":"2024-04-26T21:16:56.714671+0000","last_clean":"2024-04-26T21:16:56.714671+0000","last_became_active":"2024-04-26T21:16:56.714209+0000","last_became_peered":"2024-04-26T21:16:56.714209+0000","last_unstale":"2024-04-26T21:16:56.714671+0000","last_undegraded":"2024-04-26T21:16:56.714671+0000","last_fullsized":"2024-04-26T21:16:56.714671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:44.790915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363442+0000","last_change":"2024-04-26T21:16:03.277219+0000","last_active":"2024-04-26T21:16:56.363442+0000","last_peered":"2024-04-26T21:16:56.363442+0000","last_clean":"2024-04-26T21:16:56.363442+0000","last_became_active":"2024-04-26T21:16:03.276885+0000","last_became_peered":"2024-04-26T21:16:03.276885+0000","last_unstale":"2024-04-26T21:16:56.363442+0000","last_undegraded":"2024-04-26T21:16:56.363442+0000","last_fullsized":"2024-04-26T21:16:56.363442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:45.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362963+0000","last_change":"2024-04-26T21:16:03.703453+0000","last_active":"2024-04-26T21:16:56.362963+0000","last_peered":"2024-04-26T21:16:56.362963+0000","last_clean":"2024-04-26T21:16:56.362963+0000","last_became_active":"2024-04-26T21:16:03.703161+0000","last_became_peered":"2024-04-26T21:16:03.703161+0000","last_unstale":"2024-04-26T21:16:56.362963+0000","last_undegraded":"2024-04-26T21:16:56.362963+0000","last_fullsized":"2024-04-26T21:16:56.362963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.999243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712837+0000","last_change":"2024-04-26T21:16:03.274815+0000","last_active":"2024-04-26T21:16:56.712837+0000","last_peered":"2024-04-26T21:16:56.712837+0000","last_clean":"2024-04-26T21:16:56.712837+0000","last_became_active":"2024-04-26T21:16:03.274509+0000","last_became_peered":"2024-04-26T21:16:03.274509+0000","last_unstale":"2024-04-26T21:16:56.712837+0000","last_undegraded":"2024-04-26T21:16:56.712837+0000","last_fullsized":"2024-04-26T21:16:56.712837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:27.212714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.371100+0000","last_change":"2024-04-26T21:16:56.371100+0000","last_active":"2024-04-26T21:16:56.371100+0000","last_peered":"2024-04-26T21:16:56.371100+0000","last_clean":"2024-04-26T21:16:56.371100+0000","last_became_active":"2024-04-26T21:16:56.370711+0000","last_became_peered":"2024-04-26T21:16:56.370711+0000","last_unstale":"2024-04-26T21:16:56.371100+0000","last_undegraded":"2024-04-26T21:16:56.371100+0000","last_fullsized":"2024-04-26T21:16:56.371100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:00.143815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360698+0000","last_change":"2024-04-26T21:16:03.270417+0000","last_active":"2024-04-26T21:16:56.360698+0000","last_peered":"2024-04-26T21:16:56.360698+0000","last_clean":"2024-04-26T21:16:56.360698+0000","last_became_active":"2024-04-26T21:16:03.270066+0000","last_became_peered":"2024-04-26T21:16:03.270066+0000","last_unstale":"2024-04-26T21:16:56.360698+0000","last_undegraded":"2024-04-26T21:16:56.360698+0000","last_fullsized":"2024-04-26T21:16:56.360698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:46.794698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363117+0000","last_change":"2024-04-26T21:16:03.700221+0000","last_active":"2024-04-26T21:16:56.363117+0000","last_peered":"2024-04-26T21:16:56.363117+0000","last_clean":"2024-04-26T21:16:56.363117+0000","last_became_active":"2024-04-26T21:16:03.699798+0000","last_became_peered":"2024-04-26T21:16:03.699798+0000","last_unstale":"2024-04-26T21:16:56.363117+0000","last_undegraded":"2024-04-26T21:16:56.363117+0000","last_fullsized":"2024-04-26T21:16:56.363117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:52.083700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362074+0000","last_change":"2024-04-26T21:16:03.282355+0000","last_active":"2024-04-26T21:16:56.362074+0000","last_peered":"2024-04-26T21:16:56.362074+0000","last_clean":"2024-04-26T21:16:56.362074+0000","last_became_active":"2024-04-26T21:16:03.281169+0000","last_became_peered":"2024-04-26T21:16:03.281169+0000","last_unstale":"2024-04-26T21:16:56.362074+0000","last_undegraded":"2024-04-26T21:16:56.362074+0000","last_fullsized":"2024-04-26T21:16:56.362074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:55.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714327+0000","last_change":"2024-04-26T21:16:56.714327+0000","last_active":"2024-04-26T21:16:56.714327+0000","last_peered":"2024-04-26T21:16:56.714327+0000","last_clean":"2024-04-26T21:16:56.714327+0000","last_became_active":"2024-04-26T21:16:56.713967+0000","last_became_peered":"2024-04-26T21:16:56.713967+0000","last_unstale":"2024-04-26T21:16:56.714327+0000","last_undegraded":"2024-04-26T21:16:56.714327+0000","last_fullsized":"2024-04-26T21:16:56.714327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:32.475044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713294+0000","last_change":"2024-04-26T21:16:03.284948+0000","last_active":"2024-04-26T21:16:56.713294+0000","last_peered":"2024-04-26T21:16:56.713294+0000","last_clean":"2024-04-26T21:16:56.713294+0000","last_became_active":"2024-04-26T21:16:03.284570+0000","last_became_peered":"2024-04-26T21:16:03.284570+0000","last_unstale":"2024-04-26T21:16:56.713294+0000","last_undegraded":"2024-04-26T21:16:56.713294+0000","last_fullsized":"2024-04-26T21:16:56.713294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:22.883834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713580+0000","last_change":"2024-04-26T21:16:03.710359+0000","last_active":"2024-04-26T21:16:56.713580+0000","last_peered":"2024-04-26T21:16:56.713580+0000","last_clean":"2024-04-26T21:16:56.713580+0000","last_became_active":"2024-04-26T21:16:03.710053+0000","last_became_peered":"2024-04-26T21:16:03.710053+0000","last_unstale":"2024-04-26T21:16:56.713580+0000","last_undegraded":"2024-04-26T21:16:56.713580+0000","last_fullsized":"2024-04-26T21:16:56.713580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:45.981083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361874+0000","last_change":"2024-04-26T21:16:03.278787+0000","last_active":"2024-04-26T21:16:56.361874+0000","last_peered":"2024-04-26T21:16:56.361874+0000","last_clean":"2024-04-26T21:16:56.361874+0000","last_became_active":"2024-04-26T21:16:03.278483+0000","last_became_peered":"2024-04-26T21:16:03.278483+0000","last_unstale":"2024-04-26T21:16:56.361874+0000","last_undegraded":"2024-04-26T21:16:56.361874+0000","last_fullsized":"2024-04-26T21:16:56.361874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:40.303372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372769+0000","last_change":"2024-04-26T21:16:56.372769+0000","last_active":"2024-04-26T21:16:56.372769+0000","last_peered":"2024-04-26T21:16:56.372769+0000","last_clean":"2024-04-26T21:16:56.372769+0000","last_became_active":"2024-04-26T21:16:56.372393+0000","last_became_peered":"2024-04-26T21:16:56.372393+0000","last_unstale":"2024-04-26T21:16:56.372769+0000","last_undegraded":"2024-04-26T21:16:56.372769+0000","last_fullsized":"2024-04-26T21:16:56.372769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:29.818635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364174+0000","last_change":"2024-04-26T21:16:03.277387+0000","last_active":"2024-04-26T21:16:56.364174+0000","last_peered":"2024-04-26T21:16:56.364174+0000","last_clean":"2024-04-26T21:16:56.364174+0000","last_became_active":"2024-04-26T21:16:03.274765+0000","last_became_peered":"2024-04-26T21:16:03.274765+0000","last_unstale":"2024-04-26T21:16:56.364174+0000","last_undegraded":"2024-04-26T21:16:56.364174+0000","last_fullsized":"2024-04-26T21:16:56.364174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:51.081883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713473+0000","last_change":"2024-04-26T21:16:03.710977+0000","last_active":"2024-04-26T21:16:56.713473+0000","last_peered":"2024-04-26T21:16:56.713473+0000","last_clean":"2024-04-26T21:16:56.713473+0000","last_became_active":"2024-04-26T21:16:03.710284+0000","last_became_peered":"2024-04-26T21:16:03.710284+0000","last_unstale":"2024-04-26T21:16:56.713473+0000","last_undegraded":"2024-04-26T21:16:56.713473+0000","last_fullsized":"2024-04-26T21:16:56.713473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:46.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'25","reported_seq":201,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:17:21.803024+0000","last_change":"2024-04-26T21:16:03.276865+0000","last_active":"2024-04-26T21:17:21.803024+0000","last_peered":"2024-04-26T21:17:21.803024+0000","last_clean":"2024-04-26T21:17:21.803024+0000","last_became_active":"2024-04-26T21:16:03.276564+0000","last_became_peered":"2024-04-26T21:16:03.276564+0000","last_unstale":"2024-04-26T21:17:21.803024+0000","last_undegraded":"2024-04-26T21:17:21.803024+0000","last_fullsized":"2024-04-26T21:17:21.803024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:33.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52532,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":157,"num_read_kb":129,"num_write":25,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362088+0000","last_change":"2024-04-26T21:16:03.700441+0000","last_active":"2024-04-26T21:16:56.362088+0000","last_peered":"2024-04-26T21:16:56.362088+0000","last_clean":"2024-04-26T21:16:56.362088+0000","last_became_active":"2024-04-26T21:16:03.700142+0000","last_became_peered":"2024-04-26T21:16:03.700142+0000","last_unstale":"2024-04-26T21:16:56.362088+0000","last_undegraded":"2024-04-26T21:16:56.362088+0000","last_fullsized":"2024-04-26T21:16:56.362088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:35.166603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361872+0000","last_change":"2024-04-26T21:16:03.280144+0000","last_active":"2024-04-26T21:16:56.361872+0000","last_peered":"2024-04-26T21:16:56.361872+0000","last_clean":"2024-04-26T21:16:56.361872+0000","last_became_active":"2024-04-26T21:16:03.279844+0000","last_became_peered":"2024-04-26T21:16:03.279844+0000","last_unstale":"2024-04-26T21:16:56.361872+0000","last_undegraded":"2024-04-26T21:16:56.361872+0000","last_fullsized":"2024-04-26T21:16:56.361872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:00.704181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361493+0000","last_change":"2024-04-26T21:16:03.701732+0000","last_active":"2024-04-26T21:16:56.361493+0000","last_peered":"2024-04-26T21:16:56.361493+0000","last_clean":"2024-04-26T21:16:56.361493+0000","last_became_active":"2024-04-26T21:16:03.701424+0000","last_became_peered":"2024-04-26T21:16:03.701424+0000","last_unstale":"2024-04-26T21:16:56.361493+0000","last_undegraded":"2024-04-26T21:16:56.361493+0000","last_fullsized":"2024-04-26T21:16:56.361493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:02.293086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363077+0000","last_change":"2024-04-26T21:16:03.276816+0000","last_active":"2024-04-26T21:16:56.363077+0000","last_peered":"2024-04-26T21:16:56.363077+0000","last_clean":"2024-04-26T21:16:56.363077+0000","last_became_active":"2024-04-26T21:16:03.276471+0000","last_became_peered":"2024-04-26T21:16:03.276471+0000","last_unstale":"2024-04-26T21:16:56.363077+0000","last_undegraded":"2024-04-26T21:16:56.363077+0000","last_fullsized":"2024-04-26T21:16:56.363077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:44.630638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361866+0000","last_change":"2024-04-26T21:16:03.701002+0000","last_active":"2024-04-26T21:16:56.361866+0000","last_peered":"2024-04-26T21:16:56.361866+0000","last_clean":"2024-04-26T21:16:56.361866+0000","last_became_active":"2024-04-26T21:16:03.700649+0000","last_became_peered":"2024-04-26T21:16:03.700649+0000","last_unstale":"2024-04-26T21:16:56.361866+0000","last_undegraded":"2024-04-26T21:16:56.361866+0000","last_fullsized":"2024-04-26T21:16:56.361866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:45.571735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359679+0000","last_change":"2024-04-26T21:16:03.273727+0000","last_active":"2024-04-26T21:16:56.359679+0000","last_peered":"2024-04-26T21:16:56.359679+0000","last_clean":"2024-04-26T21:16:56.359679+0000","last_became_active":"2024-04-26T21:16:03.273287+0000","last_became_peered":"2024-04-26T21:16:03.273287+0000","last_unstale":"2024-04-26T21:16:56.359679+0000","last_undegraded":"2024-04-26T21:16:56.359679+0000","last_fullsized":"2024-04-26T21:16:56.359679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:48.713646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361559+0000","last_change":"2024-04-26T21:16:03.696760+0000","last_active":"2024-04-26T21:16:56.361559+0000","last_peered":"2024-04-26T21:16:56.361559+0000","last_clean":"2024-04-26T21:16:56.361559+0000","last_became_active":"2024-04-26T21:16:03.696446+0000","last_became_peered":"2024-04-26T21:16:03.696446+0000","last_unstale":"2024-04-26T21:16:56.361559+0000","last_undegraded":"2024-04-26T21:16:56.361559+0000","last_fullsized":"2024-04-26T21:16:56.361559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:30.710360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361962+0000","last_change":"2024-04-26T21:16:03.282643+0000","last_active":"2024-04-26T21:16:56.361962+0000","last_peered":"2024-04-26T21:16:56.361962+0000","last_clean":"2024-04-26T21:16:56.361962+0000","last_became_active":"2024-04-26T21:16:03.282315+0000","last_became_peered":"2024-04-26T21:16:03.282315+0000","last_unstale":"2024-04-26T21:16:56.361962+0000","last_undegraded":"2024-04-26T21:16:56.361962+0000","last_fullsized":"2024-04-26T21:16:56.361962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:13.358002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362813+0000","last_change":"2024-04-26T21:16:03.711992+0000","last_active":"2024-04-26T21:16:56.362813+0000","last_peered":"2024-04-26T21:16:56.362813+0000","last_clean":"2024-04-26T21:16:56.362813+0000","last_became_active":"2024-04-26T21:16:03.711755+0000","last_became_peered":"2024-04-26T21:16:03.711755+0000","last_unstale":"2024-04-26T21:16:56.362813+0000","last_undegraded":"2024-04-26T21:16:56.362813+0000","last_fullsized":"2024-04-26T21:16:56.362813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:01.321530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362836+0000","last_change":"2024-04-26T21:16:03.274722+0000","last_active":"2024-04-26T21:16:56.362836+0000","last_peered":"2024-04-26T21:16:56.362836+0000","last_clean":"2024-04-26T21:16:56.362836+0000","last_became_active":"2024-04-26T21:16:03.274399+0000","last_became_peered":"2024-04-26T21:16:03.274399+0000","last_unstale":"2024-04-26T21:16:56.362836+0000","last_undegraded":"2024-04-26T21:16:56.362836+0000","last_fullsized":"2024-04-26T21:16:56.362836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:35.247456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359442+0000","last_change":"2024-04-26T21:16:03.698344+0000","last_active":"2024-04-26T21:16:56.359442+0000","last_peered":"2024-04-26T21:16:56.359442+0000","last_clean":"2024-04-26T21:16:56.359442+0000","last_became_active":"2024-04-26T21:16:03.698037+0000","last_became_peered":"2024-04-26T21:16:03.698037+0000","last_unstale":"2024-04-26T21:16:56.359442+0000","last_undegraded":"2024-04-26T21:16:56.359442+0000","last_fullsized":"2024-04-26T21:16:56.359442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.573421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362176+0000","last_change":"2024-04-26T21:16:03.276459+0000","last_active":"2024-04-26T21:16:56.362176+0000","last_peered":"2024-04-26T21:16:56.362176+0000","last_clean":"2024-04-26T21:16:56.362176+0000","last_became_active":"2024-04-26T21:16:03.276108+0000","last_became_peered":"2024-04-26T21:16:03.276108+0000","last_unstale":"2024-04-26T21:16:56.362176+0000","last_undegraded":"2024-04-26T21:16:56.362176+0000","last_fullsized":"2024-04-26T21:16:56.362176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.364925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712741+0000","last_change":"2024-04-26T21:16:03.696030+0000","last_active":"2024-04-26T21:16:56.712741+0000","last_peered":"2024-04-26T21:16:56.712741+0000","last_clean":"2024-04-26T21:16:56.712741+0000","last_became_active":"2024-04-26T21:16:03.695717+0000","last_became_peered":"2024-04-26T21:16:03.695717+0000","last_unstale":"2024-04-26T21:16:56.712741+0000","last_undegraded":"2024-04-26T21:16:56.712741+0000","last_fullsized":"2024-04-26T21:16:56.712741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:55.702216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361784+0000","last_change":"2024-04-26T21:16:03.269188+0000","last_active":"2024-04-26T21:16:56.361784+0000","last_peered":"2024-04-26T21:16:56.361784+0000","last_clean":"2024-04-26T21:16:56.361784+0000","last_became_active":"2024-04-26T21:16:03.268850+0000","last_became_peered":"2024-04-26T21:16:03.268850+0000","last_unstale":"2024-04-26T21:16:56.361784+0000","last_undegraded":"2024-04-26T21:16:56.361784+0000","last_fullsized":"2024-04-26T21:16:56.361784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:54.183612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713804+0000","last_change":"2024-04-26T21:16:03.700284+0000","last_active":"2024-04-26T21:16:56.713804+0000","last_peered":"2024-04-26T21:16:56.713804+0000","last_clean":"2024-04-26T21:16:56.713804+0000","last_became_active":"2024-04-26T21:16:03.699965+0000","last_became_peered":"2024-04-26T21:16:03.699965+0000","last_unstale":"2024-04-26T21:16:56.713804+0000","last_undegraded":"2024-04-26T21:16:56.713804+0000","last_fullsized":"2024-04-26T21:16:56.713804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:40.547296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362424+0000","last_change":"2024-04-26T21:16:03.701370+0000","last_active":"2024-04-26T21:16:56.362424+0000","last_peered":"2024-04-26T21:16:56.362424+0000","last_clean":"2024-04-26T21:16:56.362424+0000","last_became_active":"2024-04-26T21:16:03.700585+0000","last_became_peered":"2024-04-26T21:16:03.700585+0000","last_unstale":"2024-04-26T21:16:56.362424+0000","last_undegraded":"2024-04-26T21:16:56.362424+0000","last_fullsized":"2024-04-26T21:16:56.362424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.348009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360613+0000","last_change":"2024-04-26T21:16:03.281141+0000","last_active":"2024-04-26T21:16:56.360613+0000","last_peered":"2024-04-26T21:16:56.360613+0000","last_clean":"2024-04-26T21:16:56.360613+0000","last_became_active":"2024-04-26T21:16:03.280812+0000","last_became_peered":"2024-04-26T21:16:03.280812+0000","last_unstale":"2024-04-26T21:16:56.360613+0000","last_undegraded":"2024-04-26T21:16:56.360613+0000","last_fullsized":"2024-04-26T21:16:56.360613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:11.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362156+0000","last_change":"2024-04-26T21:16:03.711312+0000","last_active":"2024-04-26T21:16:56.362156+0000","last_peered":"2024-04-26T21:16:56.362156+0000","last_clean":"2024-04-26T21:16:56.362156+0000","last_became_active":"2024-04-26T21:16:03.711026+0000","last_became_peered":"2024-04-26T21:16:03.711026+0000","last_unstale":"2024-04-26T21:16:56.362156+0000","last_undegraded":"2024-04-26T21:16:56.362156+0000","last_fullsized":"2024-04-26T21:16:56.362156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:10.185877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359923+0000","last_change":"2024-04-26T21:16:03.273428+0000","last_active":"2024-04-26T21:16:56.359923+0000","last_peered":"2024-04-26T21:16:56.359923+0000","last_clean":"2024-04-26T21:16:56.359923+0000","last_became_active":"2024-04-26T21:16:03.272854+0000","last_became_peered":"2024-04-26T21:16:03.272854+0000","last_unstale":"2024-04-26T21:16:56.359923+0000","last_undegraded":"2024-04-26T21:16:56.359923+0000","last_fullsized":"2024-04-26T21:16:56.359923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:48.735059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361618+0000","last_change":"2024-04-26T21:14:50.729892+0000","last_active":"2024-04-26T21:16:56.361618+0000","last_peered":"2024-04-26T21:16:56.361618+0000","last_clean":"2024-04-26T21:16:56.361618+0000","last_became_active":"2024-04-26T21:14:50.729430+0000","last_became_peered":"2024-04-26T21:14:50.729430+0000","last_unstale":"2024-04-26T21:16:56.361618+0000","last_undegraded":"2024-04-26T21:16:56.361618+0000","last_fullsized":"2024-04-26T21:16:56.361618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_clean_scrub_stamp":"2024-04-26T21:11:22.039289+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:13.510971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360509+0000","last_change":"2024-04-26T21:16:03.700702+0000","last_active":"2024-04-26T21:16:56.360509+0000","last_peered":"2024-04-26T21:16:56.360509+0000","last_clean":"2024-04-26T21:16:56.360509+0000","last_became_active":"2024-04-26T21:16:03.700412+0000","last_became_peered":"2024-04-26T21:16:03.700412+0000","last_unstale":"2024-04-26T21:16:56.360509+0000","last_undegraded":"2024-04-26T21:16:56.360509+0000","last_fullsized":"2024-04-26T21:16:56.360509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:49.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361824+0000","last_change":"2024-04-26T21:16:03.271730+0000","last_active":"2024-04-26T21:16:56.361824+0000","last_peered":"2024-04-26T21:16:56.361824+0000","last_clean":"2024-04-26T21:16:56.361824+0000","last_became_active":"2024-04-26T21:16:03.271401+0000","last_became_peered":"2024-04-26T21:16:03.271401+0000","last_unstale":"2024-04-26T21:16:56.361824+0000","last_undegraded":"2024-04-26T21:16:56.361824+0000","last_fullsized":"2024-04-26T21:16:56.361824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:16.469022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362223+0000","last_change":"2024-04-26T21:16:03.276229+0000","last_active":"2024-04-26T21:16:56.362223+0000","last_peered":"2024-04-26T21:16:56.362223+0000","last_clean":"2024-04-26T21:16:56.362223+0000","last_became_active":"2024-04-26T21:16:03.275910+0000","last_became_peered":"2024-04-26T21:16:03.275910+0000","last_unstale":"2024-04-26T21:16:56.362223+0000","last_undegraded":"2024-04-26T21:16:56.362223+0000","last_fullsized":"2024-04-26T21:16:56.362223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:03.932254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361647+0000","last_change":"2024-04-26T21:16:03.698293+0000","last_active":"2024-04-26T21:16:56.361647+0000","last_peered":"2024-04-26T21:16:56.361647+0000","last_clean":"2024-04-26T21:16:56.361647+0000","last_became_active":"2024-04-26T21:16:03.697935+0000","last_became_peered":"2024-04-26T21:16:03.697935+0000","last_unstale":"2024-04-26T21:16:56.361647+0000","last_undegraded":"2024-04-26T21:16:56.361647+0000","last_fullsized":"2024-04-26T21:16:56.361647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:54.292921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362587+0000","last_change":"2024-04-26T21:16:03.275162+0000","last_active":"2024-04-26T21:16:56.362587+0000","last_peered":"2024-04-26T21:16:56.362587+0000","last_clean":"2024-04-26T21:16:56.362587+0000","last_became_active":"2024-04-26T21:16:03.274791+0000","last_became_peered":"2024-04-26T21:16:03.274791+0000","last_unstale":"2024-04-26T21:16:56.362587+0000","last_undegraded":"2024-04-26T21:16:56.362587+0000","last_fullsized":"2024-04-26T21:16:56.362587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:15.583747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364592+0000","last_change":"2024-04-26T21:16:03.701473+0000","last_active":"2024-04-26T21:16:56.364592+0000","last_peered":"2024-04-26T21:16:56.364592+0000","last_clean":"2024-04-26T21:16:56.364592+0000","last_became_active":"2024-04-26T21:16:03.701070+0000","last_became_peered":"2024-04-26T21:16:03.701070+0000","last_unstale":"2024-04-26T21:16:56.364592+0000","last_undegraded":"2024-04-26T21:16:56.364592+0000","last_fullsized":"2024-04-26T21:16:56.364592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:25.373301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360117+0000","last_change":"2024-04-26T21:16:03.271036+0000","last_active":"2024-04-26T21:16:56.360117+0000","last_peered":"2024-04-26T21:16:56.360117+0000","last_clean":"2024-04-26T21:16:56.360117+0000","last_became_active":"2024-04-26T21:16:03.270686+0000","last_became_peered":"2024-04-26T21:16:03.270686+0000","last_unstale":"2024-04-26T21:16:56.360117+0000","last_undegraded":"2024-04-26T21:16:56.360117+0000","last_fullsized":"2024-04-26T21:16:56.360117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:21.025757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368114+0000","last_change":"2024-04-26T21:16:03.705314+0000","last_active":"2024-04-26T21:16:56.368114+0000","last_peered":"2024-04-26T21:16:56.368114+0000","last_clean":"2024-04-26T21:16:56.368114+0000","last_became_active":"2024-04-26T21:16:03.704952+0000","last_became_peered":"2024-04-26T21:16:03.704952+0000","last_unstale":"2024-04-26T21:16:56.368114+0000","last_undegraded":"2024-04-26T21:16:56.368114+0000","last_fullsized":"2024-04-26T21:16:56.368114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:57.363909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713329+0000","last_change":"2024-04-26T21:16:03.280995+0000","last_active":"2024-04-26T21:16:56.713329+0000","last_peered":"2024-04-26T21:16:56.713329+0000","last_clean":"2024-04-26T21:16:56.713329+0000","last_became_active":"2024-04-26T21:16:03.280615+0000","last_became_peered":"2024-04-26T21:16:03.280615+0000","last_unstale":"2024-04-26T21:16:56.713329+0000","last_undegraded":"2024-04-26T21:16:56.713329+0000","last_fullsized":"2024-04-26T21:16:56.713329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:32.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362097+0000","last_change":"2024-04-26T21:16:03.701218+0000","last_active":"2024-04-26T21:16:56.362097+0000","last_peered":"2024-04-26T21:16:56.362097+0000","last_clean":"2024-04-26T21:16:56.362097+0000","last_became_active":"2024-04-26T21:16:03.700895+0000","last_became_peered":"2024-04-26T21:16:03.700895+0000","last_unstale":"2024-04-26T21:16:56.362097+0000","last_undegraded":"2024-04-26T21:16:56.362097+0000","last_fullsized":"2024-04-26T21:16:56.362097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:44.104683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362857+0000","last_change":"2024-04-26T21:16:03.277235+0000","last_active":"2024-04-26T21:16:56.362857+0000","last_peered":"2024-04-26T21:16:56.362857+0000","last_clean":"2024-04-26T21:16:56.362857+0000","last_became_active":"2024-04-26T21:16:03.276901+0000","last_became_peered":"2024-04-26T21:16:03.276901+0000","last_unstale":"2024-04-26T21:16:56.362857+0000","last_undegraded":"2024-04-26T21:16:56.362857+0000","last_fullsized":"2024-04-26T21:16:56.362857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:16.297883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359135+0000","last_change":"2024-04-26T21:16:03.704318+0000","last_active":"2024-04-26T21:16:56.359135+0000","last_peered":"2024-04-26T21:16:56.359135+0000","last_clean":"2024-04-26T21:16:56.359135+0000","last_became_active":"2024-04-26T21:16:03.703907+0000","last_became_peered":"2024-04-26T21:16:03.703907+0000","last_unstale":"2024-04-26T21:16:56.359135+0000","last_undegraded":"2024-04-26T21:16:56.359135+0000","last_fullsized":"2024-04-26T21:16:56.359135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:16.431246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372105+0000","last_change":"2024-04-26T21:16:56.371135+0000","last_active":"2024-04-26T21:16:56.372105+0000","last_peered":"2024-04-26T21:16:56.372105+0000","last_clean":"2024-04-26T21:16:56.372105+0000","last_became_active":"2024-04-26T21:16:56.370666+0000","last_became_peered":"2024-04-26T21:16:56.370666+0000","last_unstale":"2024-04-26T21:16:56.372105+0000","last_undegraded":"2024-04-26T21:16:56.372105+0000","last_fullsized":"2024-04-26T21:16:56.372105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:05.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362114+0000","last_change":"2024-04-26T21:16:03.705026+0000","last_active":"2024-04-26T21:16:56.362114+0000","last_peered":"2024-04-26T21:16:56.362114+0000","last_clean":"2024-04-26T21:16:56.362114+0000","last_became_active":"2024-04-26T21:16:03.703336+0000","last_became_peered":"2024-04-26T21:16:03.703336+0000","last_unstale":"2024-04-26T21:16:56.362114+0000","last_undegraded":"2024-04-26T21:16:56.362114+0000","last_fullsized":"2024-04-26T21:16:56.362114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:11.746362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362335+0000","last_change":"2024-04-26T21:16:03.279263+0000","last_active":"2024-04-26T21:16:56.362335+0000","last_peered":"2024-04-26T21:16:56.362335+0000","last_clean":"2024-04-26T21:16:56.362335+0000","last_became_active":"2024-04-26T21:16:03.278948+0000","last_became_peered":"2024-04-26T21:16:03.278948+0000","last_unstale":"2024-04-26T21:16:56.362335+0000","last_undegraded":"2024-04-26T21:16:56.362335+0000","last_fullsized":"2024-04-26T21:16:56.362335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:39.078346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362036+0000","last_change":"2024-04-26T21:16:03.702913+0000","last_active":"2024-04-26T21:16:56.362036+0000","last_peered":"2024-04-26T21:16:56.362036+0000","last_clean":"2024-04-26T21:16:56.362036+0000","last_became_active":"2024-04-26T21:16:03.702507+0000","last_became_peered":"2024-04-26T21:16:03.702507+0000","last_unstale":"2024-04-26T21:16:56.362036+0000","last_undegraded":"2024-04-26T21:16:56.362036+0000","last_fullsized":"2024-04-26T21:16:56.362036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:51.846983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364220+0000","last_change":"2024-04-26T21:16:03.281898+0000","last_active":"2024-04-26T21:16:56.364220+0000","last_peered":"2024-04-26T21:16:56.364220+0000","last_clean":"2024-04-26T21:16:56.364220+0000","last_became_active":"2024-04-26T21:16:03.281544+0000","last_became_peered":"2024-04-26T21:16:03.281544+0000","last_unstale":"2024-04-26T21:16:56.364220+0000","last_undegraded":"2024-04-26T21:16:56.364220+0000","last_fullsized":"2024-04-26T21:16:56.364220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:39.061050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362089+0000","last_change":"2024-04-26T21:16:03.703006+0000","last_active":"2024-04-26T21:16:56.362089+0000","last_peered":"2024-04-26T21:16:56.362089+0000","last_clean":"2024-04-26T21:16:56.362089+0000","last_became_active":"2024-04-26T21:16:03.702705+0000","last_became_peered":"2024-04-26T21:16:03.702705+0000","last_unstale":"2024-04-26T21:16:56.362089+0000","last_undegraded":"2024-04-26T21:16:56.362089+0000","last_fullsized":"2024-04-26T21:16:56.362089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:11.885993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364037+0000","last_change":"2024-04-26T21:16:03.280891+0000","last_active":"2024-04-26T21:16:56.364037+0000","last_peered":"2024-04-26T21:16:56.364037+0000","last_clean":"2024-04-26T21:16:56.364037+0000","last_became_active":"2024-04-26T21:16:03.280577+0000","last_became_peered":"2024-04-26T21:16:03.280577+0000","last_unstale":"2024-04-26T21:16:56.364037+0000","last_undegraded":"2024-04-26T21:16:56.364037+0000","last_fullsized":"2024-04-26T21:16:56.364037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:28.698997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.378258+0000","last_change":"2024-04-26T21:16:56.378258+0000","last_active":"2024-04-26T21:16:56.378258+0000","last_peered":"2024-04-26T21:16:56.378258+0000","last_clean":"2024-04-26T21:16:56.378258+0000","last_became_active":"2024-04-26T21:16:56.371874+0000","last_became_peered":"2024-04-26T21:16:56.371874+0000","last_unstale":"2024-04-26T21:16:56.378258+0000","last_undegraded":"2024-04-26T21:16:56.378258+0000","last_fullsized":"2024-04-26T21:16:56.378258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:56.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360941+0000","last_change":"2024-04-26T21:16:03.276651+0000","last_active":"2024-04-26T21:16:56.360941+0000","last_peered":"2024-04-26T21:16:56.360941+0000","last_clean":"2024-04-26T21:16:56.360941+0000","last_became_active":"2024-04-26T21:16:03.276347+0000","last_became_peered":"2024-04-26T21:16:03.276347+0000","last_unstale":"2024-04-26T21:16:56.360941+0000","last_undegraded":"2024-04-26T21:16:56.360941+0000","last_fullsized":"2024-04-26T21:16:56.360941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:48.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362100+0000","last_change":"2024-04-26T21:16:03.705647+0000","last_active":"2024-04-26T21:16:56.362100+0000","last_peered":"2024-04-26T21:16:56.362100+0000","last_clean":"2024-04-26T21:16:56.362100+0000","last_became_active":"2024-04-26T21:16:03.705251+0000","last_became_peered":"2024-04-26T21:16:03.705251+0000","last_unstale":"2024-04-26T21:16:56.362100+0000","last_undegraded":"2024-04-26T21:16:56.362100+0000","last_fullsized":"2024-04-26T21:16:56.362100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:17.427221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359241+0000","last_change":"2024-04-26T21:16:03.701759+0000","last_active":"2024-04-26T21:16:56.359241+0000","last_peered":"2024-04-26T21:16:56.359241+0000","last_clean":"2024-04-26T21:16:56.359241+0000","last_became_active":"2024-04-26T21:16:03.701382+0000","last_became_peered":"2024-04-26T21:16:03.701382+0000","last_unstale":"2024-04-26T21:16:56.359241+0000","last_undegraded":"2024-04-26T21:16:56.359241+0000","last_fullsized":"2024-04-26T21:16:56.359241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.789455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372345+0000","last_change":"2024-04-26T21:16:56.372345+0000","last_active":"2024-04-26T21:16:56.372345+0000","last_peered":"2024-04-26T21:16:56.372345+0000","last_clean":"2024-04-26T21:16:56.372345+0000","last_became_active":"2024-04-26T21:16:56.371900+0000","last_became_peered":"2024-04-26T21:16:56.371900+0000","last_unstale":"2024-04-26T21:16:56.372345+0000","last_undegraded":"2024-04-26T21:16:56.372345+0000","last_fullsized":"2024-04-26T21:16:56.372345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:42.192142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361562+0000","last_change":"2024-04-26T21:16:03.702012+0000","last_active":"2024-04-26T21:16:56.361562+0000","last_peered":"2024-04-26T21:16:56.361562+0000","last_clean":"2024-04-26T21:16:56.361562+0000","last_became_active":"2024-04-26T21:16:03.701066+0000","last_became_peered":"2024-04-26T21:16:03.701066+0000","last_unstale":"2024-04-26T21:16:56.361562+0000","last_undegraded":"2024-04-26T21:16:56.361562+0000","last_fullsized":"2024-04-26T21:16:56.361562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:45.337320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367378+0000","last_change":"2024-04-26T21:16:03.702640+0000","last_active":"2024-04-26T21:16:56.367378+0000","last_peered":"2024-04-26T21:16:56.367378+0000","last_clean":"2024-04-26T21:16:56.367378+0000","last_became_active":"2024-04-26T21:16:03.702339+0000","last_became_peered":"2024-04-26T21:16:03.702339+0000","last_unstale":"2024-04-26T21:16:56.367378+0000","last_undegraded":"2024-04-26T21:16:56.367378+0000","last_fullsized":"2024-04-26T21:16:56.367378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:24.083734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361731+0000","last_change":"2024-04-26T21:16:03.708369+0000","last_active":"2024-04-26T21:16:56.361731+0000","last_peered":"2024-04-26T21:16:56.361731+0000","last_clean":"2024-04-26T21:16:56.361731+0000","last_became_active":"2024-04-26T21:16:03.708050+0000","last_became_peered":"2024-04-26T21:16:03.708050+0000","last_unstale":"2024-04-26T21:16:56.361731+0000","last_undegraded":"2024-04-26T21:16:56.361731+0000","last_fullsized":"2024-04-26T21:16:56.361731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:58.200043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359681+0000","last_change":"2024-04-26T21:16:03.700362+0000","last_active":"2024-04-26T21:16:56.359681+0000","last_peered":"2024-04-26T21:16:56.359681+0000","last_clean":"2024-04-26T21:16:56.359681+0000","last_became_active":"2024-04-26T21:16:03.700053+0000","last_became_peered":"2024-04-26T21:16:03.700053+0000","last_unstale":"2024-04-26T21:16:56.359681+0000","last_undegraded":"2024-04-26T21:16:56.359681+0000","last_fullsized":"2024-04-26T21:16:56.359681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:23.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363986+0000","last_change":"2024-04-26T21:16:03.707717+0000","last_active":"2024-04-26T21:16:56.363986+0000","last_peered":"2024-04-26T21:16:56.363986+0000","last_clean":"2024-04-26T21:16:56.363986+0000","last_became_active":"2024-04-26T21:16:03.707343+0000","last_became_peered":"2024-04-26T21:16:03.707343+0000","last_unstale":"2024-04-26T21:16:56.363986+0000","last_undegraded":"2024-04-26T21:16:56.363986+0000","last_fullsized":"2024-04-26T21:16:56.363986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:46.261231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369892+0000","last_change":"2024-04-26T21:16:56.369892+0000","last_active":"2024-04-26T21:16:56.369892+0000","last_peered":"2024-04-26T21:16:56.369892+0000","last_clean":"2024-04-26T21:16:56.369892+0000","last_became_active":"2024-04-26T21:16:56.369416+0000","last_became_peered":"2024-04-26T21:16:56.369416+0000","last_unstale":"2024-04-26T21:16:56.369892+0000","last_undegraded":"2024-04-26T21:16:56.369892+0000","last_fullsized":"2024-04-26T21:16:56.369892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:45.708969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362186+0000","last_change":"2024-04-26T21:16:03.699537+0000","last_active":"2024-04-26T21:16:56.362186+0000","last_peered":"2024-04-26T21:16:56.362186+0000","last_clean":"2024-04-26T21:16:56.362186+0000","last_became_active":"2024-04-26T21:16:03.699186+0000","last_became_peered":"2024-04-26T21:16:03.699186+0000","last_unstale":"2024-04-26T21:16:56.362186+0000","last_undegraded":"2024-04-26T21:16:56.362186+0000","last_fullsized":"2024-04-26T21:16:56.362186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:34.784929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362274+0000","last_change":"2024-04-26T21:16:03.709940+0000","last_active":"2024-04-26T21:16:56.362274+0000","last_peered":"2024-04-26T21:16:56.362274+0000","last_clean":"2024-04-26T21:16:56.362274+0000","last_became_active":"2024-04-26T21:16:03.707255+0000","last_became_peered":"2024-04-26T21:16:03.707255+0000","last_unstale":"2024-04-26T21:16:56.362274+0000","last_undegraded":"2024-04-26T21:16:56.362274+0000","last_fullsized":"2024-04-26T21:16:56.362274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:57.652194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713080+0000","last_change":"2024-04-26T21:16:03.711139+0000","last_active":"2024-04-26T21:16:56.713080+0000","last_peered":"2024-04-26T21:16:56.713080+0000","last_clean":"2024-04-26T21:16:56.713080+0000","last_became_active":"2024-04-26T21:16:03.709720+0000","last_became_peered":"2024-04-26T21:16:03.709720+0000","last_unstale":"2024-04-26T21:16:56.713080+0000","last_undegraded":"2024-04-26T21:16:56.713080+0000","last_fullsized":"2024-04-26T21:16:56.713080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:10.021914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361384+0000","last_change":"2024-04-26T21:16:03.699140+0000","last_active":"2024-04-26T21:16:56.361384+0000","last_peered":"2024-04-26T21:16:56.361384+0000","last_clean":"2024-04-26T21:16:56.361384+0000","last_became_active":"2024-04-26T21:16:03.698824+0000","last_became_peered":"2024-04-26T21:16:03.698824+0000","last_unstale":"2024-04-26T21:16:56.361384+0000","last_undegraded":"2024-04-26T21:16:56.361384+0000","last_fullsized":"2024-04-26T21:16:56.361384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:55.896278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359230+0000","last_change":"2024-04-26T21:16:03.708258+0000","last_active":"2024-04-26T21:16:56.359230+0000","last_peered":"2024-04-26T21:16:56.359230+0000","last_clean":"2024-04-26T21:16:56.359230+0000","last_became_active":"2024-04-26T21:16:03.707932+0000","last_became_peered":"2024-04-26T21:16:03.707932+0000","last_unstale":"2024-04-26T21:16:56.359230+0000","last_undegraded":"2024-04-26T21:16:56.359230+0000","last_fullsized":"2024-04-26T21:16:56.359230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:56.758307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244196,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53330,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":163,"num_read_kb":135,"num_write":74,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":159990,"data_compressed":5568,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":458793,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":1102004,"data_compressed":43255,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459492,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037806,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459397,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1049860,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":458910,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":511325,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":458883,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":511359,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":459014,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":458815,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1049460,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":52532,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52566,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":52726,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:17:27.898 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T21:17:27.898 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T21:17:27.899 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T21:17:27.899 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T21:17:28.144 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:28.528 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:28 smithi129 ceph-mon[29237]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:17:28.529 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:28 smithi129 ceph-mon[29237]: pgmap v334: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:28.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:28 smithi150 ceph-mon[32290]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:17:28.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:28 smithi150 ceph-mon[32290]: pgmap v334: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:28 smithi171 ceph-mon[33478]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:17:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:28 smithi171 ceph-mon[33478]: pgmap v334: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:29.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.878+0000 7f0ba214e700 1 -- 172.21.15.129:0/1221140510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c1094d0 msgr2=0x7f0b9c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:29.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.878+0000 7f0ba214e700 1 --2- 172.21.15.129:0/1221140510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c1094d0 0x7f0b9c1098b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0099c0 tx=0x7f0b8c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.878+0000 7f0ba214e700 1 -- 172.21.15.129:0/1221140510 shutdown_connections 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.878+0000 7f0ba214e700 1 --2- 172.21.15.129:0/1221140510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b9c102330 0x7f0b9c1102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.878+0000 7f0ba214e700 1 --2- 172.21.15.129:0/1221140510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c1094d0 0x7f0b9c1098b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.878+0000 7f0ba214e700 1 --2- 172.21.15.129:0/1221140510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b9c101970 0x7f0b9c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.878+0000 7f0ba214e700 1 -- 172.21.15.129:0/1221140510 >> 172.21.15.129:0/1221140510 conn(0x7f0b9c078920 msgr2=0x7f0b9c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.878+0000 7f0ba214e700 1 -- 172.21.15.129:0/1221140510 shutdown_connections 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.878+0000 7f0ba214e700 1 -- 172.21.15.129:0/1221140510 wait complete. 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.879+0000 7f0ba214e700 1 Processor -- start 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.879+0000 7f0ba214e700 1 -- start start 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0ba214e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b9c101970 0x7f0b9c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0ba214e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c102330 0x7f0b9c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0ba214e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b9c1094d0 0x7f0b9c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:29.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0ba214e700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1135e0 con 0x7f0b9c102330 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0ba214e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c113460 con 0x7f0b9c1094d0 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0ba214e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c113760 con 0x7f0b9c101970 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9affd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c102330 0x7f0b9c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9affd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c102330 0x7f0b9c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48694/0 (socket says 172.21.15.129:48694) 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9affd700 1 -- 172.21.15.129:0/3020405030 learned_addr learned my addr 172.21.15.129:0/3020405030 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9bfff700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b9c1094d0 0x7f0b9c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9b7fe700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b9c101970 0x7f0b9c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9affd700 1 -- 172.21.15.129:0/3020405030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b9c101970 msgr2=0x7f0b9c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9affd700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b9c101970 0x7f0b9c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9affd700 1 -- 172.21.15.129:0/3020405030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b9c1094d0 msgr2=0x7f0b9c199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9affd700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b9c1094d0 0x7f0b9c199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9affd700 1 -- 172.21.15.129:0/3020405030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9c19a0e0 con 0x7f0b9c102330 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b9affd700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c102330 0x7f0b9c19fd00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000c920 tx=0x7f0b9000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:29.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b98ff9700 1 -- 172.21.15.129:0/3020405030 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90004a30 con 0x7f0b9c102330 2024-04-26T21:17:29.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9c19a3d0 con 0x7f0b9c102330 2024-04-26T21:17:29.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b98ff9700 1 -- 172.21.15.129:0/3020405030 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b900054d0 con 0x7f0b9c102330 2024-04-26T21:17:29.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.880+0000 7f0b98ff9700 1 -- 172.21.15.129:0/3020405030 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90007c70 con 0x7f0b9c102330 2024-04-26T21:17:29.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.881+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9c1a6690 con 0x7f0b9c102330 2024-04-26T21:17:29.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.881+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b9c06a110 con 0x7f0b9c102330 2024-04-26T21:17:29.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.882+0000 7f0b98ff9700 1 -- 172.21.15.129:0/3020405030 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0b90004c60 con 0x7f0b9c102330 2024-04-26T21:17:29.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.883+0000 7f0b98ff9700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0b840a3cd0 0x7f0b840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:29.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.883+0000 7f0b9b7fe700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0b840a3cd0 0x7f0b840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:29.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.883+0000 7f0b98ff9700 1 -- 172.21.15.129:0/3020405030 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0b900d0470 con 0x7f0b9c102330 2024-04-26T21:17:29.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.883+0000 7f0b9b7fe700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0b840a3cd0 0x7f0b840a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c19aac0 tx=0x7f0b8c009610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:29.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:29.887+0000 7f0b98ff9700 1 -- 172.21.15.129:0/3020405030 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0b90014030 con 0x7f0b9c102330 2024-04-26T21:17:30.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.132+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0b9c02e490 con 0x7f0b9c102330 2024-04-26T21:17:30.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.132+0000 7f0b98ff9700 1 -- 172.21.15.129:0/3020405030 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0b9009cce0 con 0x7f0b9c102330 2024-04-26T21:17:30.134 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:30.134 INFO:teuthology.orchestra.run.smithi129.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0b840a3cd0 msgr2=0x7f0b840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0b840a3cd0 0x7f0b840a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c19aac0 tx=0x7f0b8c009610 comp rx=0 tx=0).stop 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c102330 msgr2=0x7f0b9c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c102330 0x7f0b9c19fd00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000c920 tx=0x7f0b9000cc30 comp rx=0 tx=0).stop 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 shutdown_connections 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0b840a3cd0 0x7f0b840a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b9c1094d0 0x7f0b9c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b9c101970 0x7f0b9c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 --2- 172.21.15.129:0/3020405030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b9c102330 0x7f0b9c19fd00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 >> 172.21.15.129:0/3020405030 conn(0x7f0b9c078920 msgr2=0x7f0b9c1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:30.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 shutdown_connections 2024-04-26T21:17:30.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.135+0000 7f0ba214e700 1 -- 172.21.15.129:0/3020405030 wait complete. 2024-04-26T21:17:30.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:30 smithi129 ceph-mon[29237]: pgmap v335: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:30.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:30 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3020405030' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:17:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:30 smithi150 ceph-mon[32290]: pgmap v335: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:30 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3020405030' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:17:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:30 smithi171 ceph-mon[33478]: pgmap v335: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:30 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3020405030' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:17:30.862 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T21:17:30.863 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.963+0000 7f52eac1d640 1 -- 172.21.15.129:0/958027982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 msgr2=0x7f52e410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.963+0000 7f52eac1d640 1 --2- 172.21.15.129:0/958027982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 0x7f52e410aac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009ca0 tx=0x7f52dc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.963+0000 7f52eac1d640 1 -- 172.21.15.129:0/958027982 shutdown_connections 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.963+0000 7f52eac1d640 1 --2- 172.21.15.129:0/958027982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e4105da0 0x7f52e4108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.963+0000 7f52eac1d640 1 --2- 172.21.15.129:0/958027982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 0x7f52e410aac0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.963+0000 7f52eac1d640 1 --2- 172.21.15.129:0/958027982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52e4103470 0x7f52e4105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.963+0000 7f52eac1d640 1 -- 172.21.15.129:0/958027982 >> 172.21.15.129:0/958027982 conn(0x7f52e40fce20 msgr2=0x7f52e40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.963+0000 7f52eac1d640 1 -- 172.21.15.129:0/958027982 shutdown_connections 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.963+0000 7f52eac1d640 1 -- 172.21.15.129:0/958027982 wait complete. 2024-04-26T21:17:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52eac1d640 1 Processor -- start 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52eac1d640 1 -- start start 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52eac1d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e4103470 0x7f52e4102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52eac1d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52e4105da0 0x7f52e4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52eac1d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 0x7f52e4101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52eac1d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52e410e4c0 con 0x7f52e41086d0 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52eac1d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f52e410e340 con 0x7f52e4103470 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52eac1d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52e410e640 con 0x7f52e4105da0 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e91ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 0x7f52e4101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e91ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 0x7f52e4101960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48710/0 (socket says 172.21.15.129:48710) 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e91ba640 1 -- 172.21.15.129:0/2695869744 learned_addr learned my addr 172.21.15.129:0/2695869744 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e3fff640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52e4105da0 0x7f52e4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e91ba640 1 -- 172.21.15.129:0/2695869744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52e4105da0 msgr2=0x7f52e4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e91ba640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52e4105da0 0x7f52e4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e91ba640 1 -- 172.21.15.129:0/2695869744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e4103470 msgr2=0x7f52e4102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:30.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e91ba640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e4103470 0x7f52e4102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:30.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e91ba640 1 -- 172.21.15.129:0/2695869744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e4102220 con 0x7f52e41086d0 2024-04-26T21:17:30.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.964+0000 7f52e3fff640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52e4105da0 0x7f52e4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:30.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.965+0000 7f52e91ba640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 0x7f52e4101960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0097b0 tx=0x7f52dc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:30.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.965+0000 7f52e1ffb640 1 -- 172.21.15.129:0/2695869744 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc004240 con 0x7f52e41086d0 2024-04-26T21:17:30.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.965+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e41a2c20 con 0x7f52e41086d0 2024-04-26T21:17:30.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.965+0000 7f52e1ffb640 1 -- 172.21.15.129:0/2695869744 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52dc0317c0 con 0x7f52e41086d0 2024-04-26T21:17:30.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.965+0000 7f52e1ffb640 1 -- 172.21.15.129:0/2695869744 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc02fd80 con 0x7f52e41086d0 2024-04-26T21:17:30.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.965+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e41a3190 con 0x7f52e41086d0 2024-04-26T21:17:30.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.966+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e4069ba0 con 0x7f52e41086d0 2024-04-26T21:17:30.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.967+0000 7f52e1ffb640 1 -- 172.21.15.129:0/2695869744 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f52dc0043e0 con 0x7f52e41086d0 2024-04-26T21:17:30.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.968+0000 7f52e1ffb640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f52d40a3c40 0x7f52d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:30.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.968+0000 7f52e89b9640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f52d40a3c40 0x7f52d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:30.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.968+0000 7f52e1ffb640 1 -- 172.21.15.129:0/2695869744 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f52dc0f1f10 con 0x7f52e41086d0 2024-04-26T21:17:30.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.968+0000 7f52e89b9640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f52d40a3c40 0x7f52d40a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52d8005fd0 tx=0x7f52d8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:30.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:30.971+0000 7f52e1ffb640 1 -- 172.21.15.129:0/2695869744 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f52dc0be890 con 0x7f52e41086d0 2024-04-26T21:17:31.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.151+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52e402db50 con 0x7f52e41086d0 2024-04-26T21:17:31.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.152+0000 7f52e1ffb640 1 -- 172.21.15.129:0/2695869744 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7f52dc0be190 con 0x7f52e41086d0 2024-04-26T21:17:31.154 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:31.154 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f52d40a3c40 msgr2=0x7f52d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f52d40a3c40 0x7f52d40a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52d8005fd0 tx=0x7f52d8005850 comp rx=0 tx=0).stop 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 msgr2=0x7f52e4101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 0x7f52e4101960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0097b0 tx=0x7f52dc003c20 comp rx=0 tx=0).stop 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 shutdown_connections 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f52d40a3c40 0x7f52d40a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e41086d0 0x7f52e4101960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52e4105da0 0x7f52e4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 --2- 172.21.15.129:0/2695869744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e4103470 0x7f52e4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 >> 172.21.15.129:0/2695869744 conn(0x7f52e40fce20 msgr2=0x7f52e4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 shutdown_connections 2024-04-26T21:17:31.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.154+0000 7f52eac1d640 1 -- 172.21.15.129:0/2695869744 wait complete. 2024-04-26T21:17:31.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:16:01.621500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:16:02.592849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:17:31.168 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 -- 172.21.15.129:0/1505274164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc103b70 msgr2=0x7fbcfc103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 --2- 172.21.15.129:0/1505274164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc103b70 0x7fbcfc103ff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009900 tx=0x7fbcf002a410 comp rx=0 tx=0).stop 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 -- 172.21.15.129:0/1505274164 shutdown_connections 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 --2- 172.21.15.129:0/1505274164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcfc104530 0x7fbcfc10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 --2- 172.21.15.129:0/1505274164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc103b70 0x7fbcfc103ff0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 --2- 172.21.15.129:0/1505274164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcfc102980 0x7fbcfc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 -- 172.21.15.129:0/1505274164 >> 172.21.15.129:0/1505274164 conn(0x7fbcfc0fdf10 msgr2=0x7fbcfc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 -- 172.21.15.129:0/1505274164 shutdown_connections 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 -- 172.21.15.129:0/1505274164 wait complete. 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 Processor -- start 2024-04-26T21:17:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.268+0000 7fbd02122640 1 -- start start 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbd02122640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcfc102980 0x7fbcfc0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbd02122640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcfc103b70 0x7fbcfc074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbd02122640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc104530 0x7fbcfc075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbd02122640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc10e520 con 0x7fbcfc104530 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbd02122640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc10e3a0 con 0x7fbcfc102980 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbd02122640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc10e6a0 con 0x7fbcfc103b70 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc104530 0x7fbcfc075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc104530 0x7fbcfc075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48730/0 (socket says 172.21.15.129:48730) 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfbfff640 1 -- 172.21.15.129:0/1955678974 learned_addr learned my addr 172.21.15.129:0/1955678974 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfaffd640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcfc103b70 0x7fbcfc074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfb7fe640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcfc102980 0x7fbcfc0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfbfff640 1 -- 172.21.15.129:0/1955678974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcfc103b70 msgr2=0x7fbcfc074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfbfff640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcfc103b70 0x7fbcfc074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfbfff640 1 -- 172.21.15.129:0/1955678974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcfc102980 msgr2=0x7fbcfc0766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfbfff640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcfc102980 0x7fbcfc0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfbfff640 1 -- 172.21.15.129:0/1955678974 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc075af0 con 0x7fbcfc104530 2024-04-26T21:17:31.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcfbfff640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc104530 0x7fbcfc075260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400eaa0 tx=0x7fbcf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:31.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcf8ff9640 1 -- 172.21.15.129:0/1955678974 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf400cdb0 con 0x7fbcfc104530 2024-04-26T21:17:31.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc1a2d90 con 0x7fbcfc104530 2024-04-26T21:17:31.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcf8ff9640 1 -- 172.21.15.129:0/1955678974 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf4005020 con 0x7fbcfc104530 2024-04-26T21:17:31.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbcf8ff9640 1 -- 172.21.15.129:0/1955678974 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40116d0 con 0x7fbcfc104530 2024-04-26T21:17:31.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.269+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc1a32a0 con 0x7fbcfc104530 2024-04-26T21:17:31.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.270+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcfc109d70 con 0x7fbcfc104530 2024-04-26T21:17:31.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.272+0000 7fbcf8ff9640 1 -- 172.21.15.129:0/1955678974 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbcf40051d0 con 0x7fbcfc104530 2024-04-26T21:17:31.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.272+0000 7fbcf8ff9640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbce40a3c40 0x7fbce40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.272+0000 7fbcfb7fe640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbce40a3c40 0x7fbce40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.272+0000 7fbcf8ff9640 1 -- 172.21.15.129:0/1955678974 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fbcf40d0c60 con 0x7fbcfc104530 2024-04-26T21:17:31.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.273+0000 7fbcfb7fe640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbce40a3c40 0x7fbce40a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc1039d0 tx=0x7fbcec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:31.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.276+0000 7fbcf8ff9640 1 -- 172.21.15.129:0/1955678974 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbcf4016030 con 0x7fbcfc104530 2024-04-26T21:17:31.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.456+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbcfc063530 con 0x7fbcfc104530 2024-04-26T21:17:31.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.457+0000 7fbcf8ff9640 1 -- 172.21.15.129:0/1955678974 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fbcf409d4d0 con 0x7fbcfc104530 2024-04-26T21:17:31.458 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.458+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbce40a3c40 msgr2=0x7fbce40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.458+0000 7fbd02122640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbce40a3c40 0x7fbce40a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc1039d0 tx=0x7fbcec008040 comp rx=0 tx=0).stop 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.458+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc104530 msgr2=0x7fbcfc075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.458+0000 7fbd02122640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc104530 0x7fbcfc075260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400eaa0 tx=0x7fbcf400ef60 comp rx=0 tx=0).stop 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.459+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 shutdown_connections 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.459+0000 7fbd02122640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbce40a3c40 0x7fbce40a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.459+0000 7fbd02122640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbcfc104530 0x7fbcfc075260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.459+0000 7fbd02122640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcfc103b70 0x7fbcfc074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.459+0000 7fbd02122640 1 --2- 172.21.15.129:0/1955678974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcfc102980 0x7fbcfc0766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.459+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 >> 172.21.15.129:0/1955678974 conn(0x7fbcfc0fdf10 msgr2=0x7fbcfc106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.459+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 shutdown_connections 2024-04-26T21:17:31.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.459+0000 7fbd02122640 1 -- 172.21.15.129:0/1955678974 wait complete. 2024-04-26T21:17:31.469 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:31 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2695869744' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:31.471 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:17:31.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.571+0000 7fd5f80f1640 1 -- 172.21.15.129:0/911768290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0101890 msgr2=0x7fd5f0101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.571+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/911768290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0101890 0x7fd5f0101cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009900 tx=0x7fd5ec02a410 comp rx=0 tx=0).stop 2024-04-26T21:17:31.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.571+0000 7fd5f80f1640 1 -- 172.21.15.129:0/911768290 shutdown_connections 2024-04-26T21:17:31.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.571+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/911768290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f0103440 0x7fd5f0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.571+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/911768290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f0102a80 0x7fd5f0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.571+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/911768290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0101890 0x7fd5f0101cb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.571+0000 7fd5f80f1640 1 -- 172.21.15.129:0/911768290 >> 172.21.15.129:0/911768290 conn(0x7fd5f00fce00 msgr2=0x7fd5f00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.571+0000 7fd5f80f1640 1 -- 172.21.15.129:0/911768290 shutdown_connections 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.571+0000 7fd5f80f1640 1 -- 172.21.15.129:0/911768290 wait complete. 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f80f1640 1 Processor -- start 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f80f1640 1 -- start start 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f80f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f0101890 0x7fd5f006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f80f1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f0102a80 0x7fd5f019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f80f1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0103440 0x7fd5f019ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f80f1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f010d390 con 0x7fd5f0103440 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f80f1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f010d210 con 0x7fd5f0102a80 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f80f1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f010d510 con 0x7fd5f0101890 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f668e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0103440 0x7fd5f019ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f668e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0103440 0x7fd5f019ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48750/0 (socket says 172.21.15.129:48750) 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f668e640 1 -- 172.21.15.129:0/4244740383 learned_addr learned my addr 172.21.15.129:0/4244740383 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f568c640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f0102a80 0x7fd5f019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f5e8d640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f0101890 0x7fd5f006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f668e640 1 -- 172.21.15.129:0/4244740383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f0101890 msgr2=0x7fd5f006baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f668e640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f0101890 0x7fd5f006baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f668e640 1 -- 172.21.15.129:0/4244740383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f0102a80 msgr2=0x7fd5f019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f668e640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f0102a80 0x7fd5f019d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f668e640 1 -- 172.21.15.129:0/4244740383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f01a0830 con 0x7fd5f0103440 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f668e640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0103440 0x7fd5f019ff20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800cad0 tx=0x7fd5e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5e6ffd640 1 -- 172.21.15.129:0/4244740383 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800ae20 con 0x7fd5f0103440 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5e6ffd640 1 -- 172.21.15.129:0/4244740383 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e8005020 con 0x7fd5f0103440 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f01a0ac0 con 0x7fd5f0103440 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.572+0000 7fd5e6ffd640 1 -- 172.21.15.129:0/4244740383 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800f6b0 con 0x7fd5f0103440 2024-04-26T21:17:31.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.573+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f01a1010 con 0x7fd5f0103440 2024-04-26T21:17:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.573+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f0108c80 con 0x7fd5f0103440 2024-04-26T21:17:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.575+0000 7fd5e6ffd640 1 -- 172.21.15.129:0/4244740383 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd5e80051d0 con 0x7fd5f0103440 2024-04-26T21:17:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.575+0000 7fd5e6ffd640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5dc0a3cc0 0x7fd5dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.575+0000 7fd5f5e8d640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5dc0a3cc0 0x7fd5dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.575+0000 7fd5e6ffd640 1 -- 172.21.15.129:0/4244740383 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd5e80d09e0 con 0x7fd5f0103440 2024-04-26T21:17:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.576+0000 7fd5f5e8d640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5dc0a3cc0 0x7fd5dc0a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec02a930 tx=0x7fd5ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:31.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.579+0000 7fd5e6ffd640 1 -- 172.21.15.129:0/4244740383 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd5e8014030 con 0x7fd5f0103440 2024-04-26T21:17:31.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:31 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2695869744' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:31.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:31 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2695869744' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:31.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.760+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd5f002dd20 con 0x7fd5f0103440 2024-04-26T21:17:31.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.761+0000 7fd5e6ffd640 1 -- 172.21.15.129:0/4244740383 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fd5e809d2d0 con 0x7fd5f0103440 2024-04-26T21:17:31.762 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-26T21:17:31.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5dc0a3cc0 msgr2=0x7fd5dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5dc0a3cc0 0x7fd5dc0a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec02a930 tx=0x7fd5ec000bc0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0103440 msgr2=0x7fd5f019ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0103440 0x7fd5f019ff20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800cad0 tx=0x7fd5e800cf90 comp rx=0 tx=0).stop 2024-04-26T21:17:31.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 shutdown_connections 2024-04-26T21:17:31.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5dc0a3cc0 0x7fd5dc0a6180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f0102a80 0x7fd5f019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f0103440 0x7fd5f019ff20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 --2- 172.21.15.129:0/4244740383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f0101890 0x7fd5f006baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 >> 172.21.15.129:0/4244740383 conn(0x7fd5f00fce00 msgr2=0x7fd5f0105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:31.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 shutdown_connections 2024-04-26T21:17:31.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.763+0000 7fd5f80f1640 1 -- 172.21.15.129:0/4244740383 wait complete. 2024-04-26T21:17:31.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:17:31.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 -- 172.21.15.129:0/4048566041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4102a80 msgr2=0x7fa5c4102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/4048566041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4102a80 0x7fa5c4102f00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009900 tx=0x7fa5b402a410 comp rx=0 tx=0).stop 2024-04-26T21:17:31.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 -- 172.21.15.129:0/4048566041 shutdown_connections 2024-04-26T21:17:31.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/4048566041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5c4103440 0x7fa5c4109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/4048566041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4102a80 0x7fa5c4102f00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/4048566041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c4101890 0x7fa5c4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 -- 172.21.15.129:0/4048566041 >> 172.21.15.129:0/4048566041 conn(0x7fa5c40fce20 msgr2=0x7fa5c40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:31.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 -- 172.21.15.129:0/4048566041 shutdown_connections 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 -- 172.21.15.129:0/4048566041 wait complete. 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 Processor -- start 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 -- start start 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5c4101890 0x7fa5c4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.877+0000 7fa5cbc09640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c4102a80 0x7fa5c4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5cbc09640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4103440 0x7fa5c419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5cbc09640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d3c0 con 0x7fa5c4103440 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5cbc09640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d240 con 0x7fa5c4102a80 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5cbc09640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d540 con 0x7fa5c4101890 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5ca1a6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4103440 0x7fa5c419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5ca1a6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4103440 0x7fa5c419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48756/0 (socket says 172.21.15.129:48756) 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5ca1a6640 1 -- 172.21.15.129:0/1606296194 learned_addr learned my addr 172.21.15.129:0/1606296194 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5c91a4640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c4102a80 0x7fa5c4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5c99a5640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5c4101890 0x7fa5c4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5ca1a6640 1 -- 172.21.15.129:0/1606296194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5c4101890 msgr2=0x7fa5c4197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5ca1a6640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5c4101890 0x7fa5c4197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5ca1a6640 1 -- 172.21.15.129:0/1606296194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c4102a80 msgr2=0x7fa5c4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5ca1a6640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c4102a80 0x7fa5c4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5ca1a6640 1 -- 172.21.15.129:0/1606296194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c419f680 con 0x7fa5c4103440 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5ca1a6640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4103440 0x7fa5c419f010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00eb10 tx=0x7fa5bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5baffd640 1 -- 172.21.15.129:0/1606296194 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc004b20 con 0x7fa5c4103440 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5baffd640 1 -- 172.21.15.129:0/1606296194 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5bc004cc0 con 0x7fa5c4103440 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c419f970 con 0x7fa5c4103440 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5baffd640 1 -- 172.21.15.129:0/1606296194 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc010670 con 0x7fa5c4103440 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.878+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c419fee0 con 0x7fa5c4103440 2024-04-26T21:17:31.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.879+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4108bb0 con 0x7fa5c4103440 2024-04-26T21:17:31.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.881+0000 7fa5baffd640 1 -- 172.21.15.129:0/1606296194 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa5bc005000 con 0x7fa5c4103440 2024-04-26T21:17:31.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.881+0000 7fa5baffd640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:31.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.881+0000 7fa5c99a5640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:31.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.881+0000 7fa5baffd640 1 -- 172.21.15.129:0/1606296194 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa5bc0d0d20 con 0x7fa5c4103440 2024-04-26T21:17:31.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.882+0000 7fa5c99a5640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00058c0 tx=0x7fa5c0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:31.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:31.885+0000 7fa5baffd640 1 -- 172.21.15.129:0/1606296194 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa5bc054060 con 0x7fa5c4103440 2024-04-26T21:17:32.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.063+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa5c402dc60 con 0x7fa5c4103440 2024-04-26T21:17:32.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.063+0000 7fa5baffd640 1 -- 172.21.15.129:0/1606296194 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fa5bc09d590 con 0x7fa5c4103440 2024-04-26T21:17:32.064 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-26T21:17:32.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa5b00a3cc0 msgr2=0x7fa5b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00058c0 tx=0x7fa5c0005850 comp rx=0 tx=0).stop 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4103440 msgr2=0x7fa5c419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4103440 0x7fa5c419f010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00eb10 tx=0x7fa5bc00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 shutdown_connections 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c4102a80 0x7fa5c4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5c4103440 0x7fa5c419f010 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 --2- 172.21.15.129:0/1606296194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5c4101890 0x7fa5c4197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 >> 172.21.15.129:0/1606296194 conn(0x7fa5c40fce20 msgr2=0x7fa5c4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 shutdown_connections 2024-04-26T21:17:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.065+0000 7fa5cbc09640 1 -- 172.21.15.129:0/1606296194 wait complete. 2024-04-26T21:17:32.078 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:17:32.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.174+0000 7fa42d474640 1 -- 172.21.15.129:0/2838641284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 msgr2=0x7fa42810aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.174+0000 7fa42d474640 1 --2- 172.21.15.129:0/2838641284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 0x7fa42810aab0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa420009ca0 tx=0x7fa42002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.174+0000 7fa42d474640 1 -- 172.21.15.129:0/2838641284 shutdown_connections 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.174+0000 7fa42d474640 1 --2- 172.21.15.129:0/2838641284 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa428105d90 0x7fa428108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.174+0000 7fa42d474640 1 --2- 172.21.15.129:0/2838641284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 0x7fa42810aab0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.174+0000 7fa42d474640 1 --2- 172.21.15.129:0/2838641284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa428103460 0x7fa428105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.174+0000 7fa42d474640 1 -- 172.21.15.129:0/2838641284 >> 172.21.15.129:0/2838641284 conn(0x7fa4280fce10 msgr2=0x7fa4280ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.174+0000 7fa42d474640 1 -- 172.21.15.129:0/2838641284 shutdown_connections 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.174+0000 7fa42d474640 1 -- 172.21.15.129:0/2838641284 wait complete. 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa42d474640 1 Processor -- start 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa42d474640 1 -- start start 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa42d474640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa428103460 0x7fa428198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa42d474640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa428105d90 0x7fa4281990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa42d474640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 0x7fa4281a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa42d474640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa42810e4d0 con 0x7fa4281086c0 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa42d474640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa42810e350 con 0x7fa428105d90 2024-04-26T21:17:32.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa42d474640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa42810e650 con 0x7fa428103460 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa4277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 0x7fa4281a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa4277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 0x7fa4281a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48778/0 (socket says 172.21.15.129:48778) 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa4277fe640 1 -- 172.21.15.129:0/3610530501 learned_addr learned my addr 172.21.15.129:0/3610530501 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa41ffff640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa428105d90 0x7fa4281990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa426ffd640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa428103460 0x7fa428198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa4277fe640 1 -- 172.21.15.129:0/3610530501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa428103460 msgr2=0x7fa428198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa4277fe640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa428103460 0x7fa428198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa4277fe640 1 -- 172.21.15.129:0/3610530501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa428105d90 msgr2=0x7fa4281990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa4277fe640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa428105d90 0x7fa4281990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa4277fe640 1 -- 172.21.15.129:0/3610530501 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4281a07c0 con 0x7fa4281086c0 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa4277fe640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 0x7fa4281a0150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa4200097b0 tx=0x7fa420031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.175+0000 7fa424ff9640 1 -- 172.21.15.129:0/3610530501 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420003bf0 con 0x7fa4281086c0 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.176+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281a0a50 con 0x7fa4281086c0 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.176+0000 7fa424ff9640 1 -- 172.21.15.129:0/3610530501 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa420003d90 con 0x7fa4281086c0 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.176+0000 7fa424ff9640 1 -- 172.21.15.129:0/3610530501 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42002fc70 con 0x7fa4281086c0 2024-04-26T21:17:32.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.176+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4281a0fa0 con 0x7fa4281086c0 2024-04-26T21:17:32.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.176+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4280fe900 con 0x7fa4281086c0 2024-04-26T21:17:32.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.178+0000 7fa424ff9640 1 -- 172.21.15.129:0/3610530501 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa420004040 con 0x7fa4281086c0 2024-04-26T21:17:32.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.178+0000 7fa424ff9640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4140a3c40 0x7fa4140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.178+0000 7fa426ffd640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4140a3c40 0x7fa4140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.178+0000 7fa424ff9640 1 -- 172.21.15.129:0/3610530501 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa4200f1ba0 con 0x7fa4281086c0 2024-04-26T21:17:32.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.179+0000 7fa426ffd640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4140a3c40 0x7fa4140a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa418005fd0 tx=0x7fa41800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:32.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.182+0000 7fa424ff9640 1 -- 172.21.15.129:0/3610530501 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa4200be520 con 0x7fa4281086c0 2024-04-26T21:17:32.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.398+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa428063530 con 0x7fa4281086c0 2024-04-26T21:17:32.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.399+0000 7fa424ff9640 1 -- 172.21.15.129:0/3610530501 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7fa4200bde20 con 0x7fa4281086c0 2024-04-26T21:17:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:32.400 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:17:32.400 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:17:32.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4140a3c40 msgr2=0x7fa4140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4140a3c40 0x7fa4140a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa418005fd0 tx=0x7fa41800b390 comp rx=0 tx=0).stop 2024-04-26T21:17:32.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 msgr2=0x7fa4281a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 0x7fa4281a0150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa4200097b0 tx=0x7fa420031840 comp rx=0 tx=0).stop 2024-04-26T21:17:32.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 shutdown_connections 2024-04-26T21:17:32.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4140a3c40 0x7fa4140a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa428105d90 0x7fa4281990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4281086c0 0x7fa4281a0150 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 --2- 172.21.15.129:0/3610530501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa428103460 0x7fa428198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 >> 172.21.15.129:0/3610530501 conn(0x7fa4280fce10 msgr2=0x7fa428103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:32.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 shutdown_connections 2024-04-26T21:17:32.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.401+0000 7fa42d474640 1 -- 172.21.15.129:0/3610530501 wait complete. 2024-04-26T21:17:32.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:17:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.510+0000 7f42931e7640 1 -- 172.21.15.129:0/1712395173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 msgr2=0x7f428c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.510+0000 7f42931e7640 1 --2- 172.21.15.129:0/1712395173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 0x7f428c109cf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4284009ca0 tx=0x7f428402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.510+0000 7f42931e7640 1 -- 172.21.15.129:0/1712395173 shutdown_connections 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.510+0000 7f42931e7640 1 --2- 172.21.15.129:0/1712395173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 0x7f428c109cf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.510+0000 7f42931e7640 1 --2- 172.21.15.129:0/1712395173 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f428c102aa0 0x7f428c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.510+0000 7f42931e7640 1 --2- 172.21.15.129:0/1712395173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c1018b0 0x7f428c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.510+0000 7f42931e7640 1 -- 172.21.15.129:0/1712395173 >> 172.21.15.129:0/1712395173 conn(0x7f428c0fce20 msgr2=0x7f428c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 -- 172.21.15.129:0/1712395173 shutdown_connections 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 -- 172.21.15.129:0/1712395173 wait complete. 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 Processor -- start 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 -- start start 2024-04-26T21:17:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f428c1018b0 0x7f428c06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c102aa0 0x7f428c19d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 0x7f428c19ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f428c10d3c0 con 0x7f428c103460 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f428c10d240 con 0x7f428c102aa0 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f42931e7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f428c10d540 con 0x7f428c1018b0 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f4291784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 0x7f428c19ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f4291784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 0x7f428c19ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40662/0 (socket says 172.21.15.129:40662) 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f4291784640 1 -- 172.21.15.129:0/1903213247 learned_addr learned my addr 172.21.15.129:0/1903213247 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f428bfff640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c102aa0 0x7f428c19d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.511+0000 7f4290f83640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f428c1018b0 0x7f428c06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f4291784640 1 -- 172.21.15.129:0/1903213247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f428c1018b0 msgr2=0x7f428c06b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f4291784640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f428c1018b0 0x7f428c06b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f4291784640 1 -- 172.21.15.129:0/1903213247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c102aa0 msgr2=0x7f428c19d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f4291784640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c102aa0 0x7f428c19d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f4291784640 1 -- 172.21.15.129:0/1903213247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c1a07d0 con 0x7f428c103460 2024-04-26T21:17:32.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f428bfff640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c102aa0 0x7f428c19d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:32.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f4291784640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 0x7f428c19ff10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4284006010 tx=0x7f428402f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:32.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f4289ffb640 1 -- 172.21.15.129:0/1903213247 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284003cf0 con 0x7f428c103460 2024-04-26T21:17:32.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f4289ffb640 1 -- 172.21.15.129:0/1903213247 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42840044f0 con 0x7f428c103460 2024-04-26T21:17:32.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c1a0a60 con 0x7f428c103460 2024-04-26T21:17:32.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f4289ffb640 1 -- 172.21.15.129:0/1903213247 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428402fd50 con 0x7f428c103460 2024-04-26T21:17:32.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.512+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c1a1000 con 0x7f428c103460 2024-04-26T21:17:32.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.513+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c108c50 con 0x7f428c103460 2024-04-26T21:17:32.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.514+0000 7f4289ffb640 1 -- 172.21.15.129:0/1903213247 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4284004040 con 0x7f428c103460 2024-04-26T21:17:32.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.515+0000 7f4289ffb640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f427c0a3d40 0x7f427c0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.515+0000 7f4290f83640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f427c0a3d40 0x7f427c0a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.515+0000 7f4289ffb640 1 -- 172.21.15.129:0/1903213247 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f42840f1bc0 con 0x7f428c103460 2024-04-26T21:17:32.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.515+0000 7f4290f83640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f427c0a3d40 0x7f427c0a6200 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4280005fd0 tx=0x7f4280005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.519+0000 7f4289ffb640 1 -- 172.21.15.129:0/1903213247 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f42840be4e0 con 0x7f428c103460 2024-04-26T21:17:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:32 smithi150 ceph-mon[32290]: pgmap v336: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:32 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1955678974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:32 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/4244740383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:32 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1606296194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:32.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.696+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f428c02dcf0 con 0x7f428c103460 2024-04-26T21:17:32.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.698+0000 7f4289ffb640 1 -- 172.21.15.129:0/1903213247 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7f42840098b0 con 0x7f428c103460 2024-04-26T21:17:32.699 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:32.699 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:17:32.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f427c0a3d40 msgr2=0x7f427c0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f427c0a3d40 0x7f427c0a6200 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4280005fd0 tx=0x7f4280005850 comp rx=0 tx=0).stop 2024-04-26T21:17:32.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 msgr2=0x7f428c19ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 0x7f428c19ff10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4284006010 tx=0x7f428402f460 comp rx=0 tx=0).stop 2024-04-26T21:17:32.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 shutdown_connections 2024-04-26T21:17:32.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f427c0a3d40 0x7f427c0a6200 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f428c102aa0 0x7f428c19d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f428c103460 0x7f428c19ff10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 --2- 172.21.15.129:0/1903213247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f428c1018b0 0x7f428c06b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 >> 172.21.15.129:0/1903213247 conn(0x7f428c0fce20 msgr2=0x7f428c105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:32.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 shutdown_connections 2024-04-26T21:17:32.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.700+0000 7f42931e7640 1 -- 172.21.15.129:0/1903213247 wait complete. 2024-04-26T21:17:32.711 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:32 smithi129 ceph-mon[29237]: pgmap v336: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:32.712 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:32 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1955678974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:32.712 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:32 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/4244740383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:32.712 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:32 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1606296194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:32.713 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:16:01.621500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:16:02.592849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:17:32.713 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:17:32.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:32 smithi171 ceph-mon[33478]: pgmap v336: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:32.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:32 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1955678974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:32.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:32 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/4244740383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:32.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:32 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1606296194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:32.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 -- 172.21.15.129:0/758215880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738101870 msgr2=0x7fe738101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/758215880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738101870 0x7fe738101c90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009900 tx=0x7fe72c02a410 comp rx=0 tx=0).stop 2024-04-26T21:17:32.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 -- 172.21.15.129:0/758215880 shutdown_connections 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/758215880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe738102a60 0x7fe738102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/758215880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe738103420 0x7fe738109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/758215880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738101870 0x7fe738101c90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 -- 172.21.15.129:0/758215880 >> 172.21.15.129:0/758215880 conn(0x7fe7380fcde0 msgr2=0x7fe7380ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 -- 172.21.15.129:0/758215880 shutdown_connections 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 -- 172.21.15.129:0/758215880 wait complete. 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 Processor -- start 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.813+0000 7fe73e1e9640 1 -- start start 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe73e1e9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe738101870 0x7fe73806a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe73e1e9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe738102a60 0x7fe73819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe73e1e9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738103420 0x7fe73819ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe73e1e9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe73810d3a0 con 0x7fe738103420 2024-04-26T21:17:32.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe73e1e9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe73810d220 con 0x7fe738101870 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe73e1e9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe73810d520 con 0x7fe738102a60 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe737fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738103420 0x7fe73819ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe737fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738103420 0x7fe73819ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40678/0 (socket says 172.21.15.129:40678) 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe737fff640 1 -- 172.21.15.129:0/1647460806 learned_addr learned my addr 172.21.15.129:0/1647460806 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe7377fe640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe738101870 0x7fe73806a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe737fff640 1 -- 172.21.15.129:0/1647460806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe738102a60 msgr2=0x7fe73819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe737fff640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe738102a60 0x7fe73819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe737fff640 1 -- 172.21.15.129:0/1647460806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe738101870 msgr2=0x7fe73806a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe737fff640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe738101870 0x7fe73806a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe737fff640 1 -- 172.21.15.129:0/1647460806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7381a0820 con 0x7fe738103420 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe7377fe640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe738101870 0x7fe73806a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe737fff640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738103420 0x7fe73819ff10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe73000cad0 tx=0x7fe73000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe734ff9640 1 -- 172.21.15.129:0/1647460806 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73000ae20 con 0x7fe738103420 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7381a0ab0 con 0x7fe738103420 2024-04-26T21:17:32.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe734ff9640 1 -- 172.21.15.129:0/1647460806 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe730005020 con 0x7fe738103420 2024-04-26T21:17:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe734ff9640 1 -- 172.21.15.129:0/1647460806 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73000f6b0 con 0x7fe738103420 2024-04-26T21:17:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.814+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7381a1000 con 0x7fe738103420 2024-04-26T21:17:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.815+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe738108c60 con 0x7fe738103420 2024-04-26T21:17:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.816+0000 7fe734ff9640 1 -- 172.21.15.129:0/1647460806 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe7300051d0 con 0x7fe738103420 2024-04-26T21:17:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.817+0000 7fe734ff9640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7200a3c40 0x7fe7200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:32.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.817+0000 7fe7377fe640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7200a3c40 0x7fe7200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:32.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.817+0000 7fe734ff9640 1 -- 172.21.15.129:0/1647460806 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe7300d0680 con 0x7fe738103420 2024-04-26T21:17:32.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.817+0000 7fe7377fe640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7200a3c40 0x7fe7200a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe72c006010 tx=0x7fe72c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:32.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.821+0000 7fe734ff9640 1 -- 172.21.15.129:0/1647460806 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe730014030 con 0x7fe738103420 2024-04-26T21:17:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.998+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe73802dd20 con 0x7fe738103420 2024-04-26T21:17:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:32.998+0000 7fe734ff9640 1 -- 172.21.15.129:0/1647460806 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fe73009cef0 con 0x7fe738103420 2024-04-26T21:17:32.999 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:17:33.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7200a3c40 msgr2=0x7fe7200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7200a3c40 0x7fe7200a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe72c006010 tx=0x7fe72c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738103420 msgr2=0x7fe73819ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738103420 0x7fe73819ff10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe73000cad0 tx=0x7fe73000cf90 comp rx=0 tx=0).stop 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 shutdown_connections 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7200a3c40 0x7fe7200a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe738103420 0x7fe73819ff10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe738102a60 0x7fe73819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 --2- 172.21.15.129:0/1647460806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe738101870 0x7fe73806a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 >> 172.21.15.129:0/1647460806 conn(0x7fe7380fcde0 msgr2=0x7fe738105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 shutdown_connections 2024-04-26T21:17:33.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.000+0000 7fe73e1e9640 1 -- 172.21.15.129:0/1647460806 wait complete. 2024-04-26T21:17:33.013 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.111+0000 7f57dfd9d640 1 -- 172.21.15.129:0/3151549460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8102a60 msgr2=0x7f57d8102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.111+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/3151549460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8102a60 0x7f57d8102ee0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009900 tx=0x7f57c802a410 comp rx=0 tx=0).stop 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.111+0000 7f57dfd9d640 1 -- 172.21.15.129:0/3151549460 shutdown_connections 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.111+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/3151549460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8103420 0x7f57d8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.111+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/3151549460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8102a60 0x7f57d8102ee0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.111+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/3151549460 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8101870 0x7f57d8101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.111+0000 7f57dfd9d640 1 -- 172.21.15.129:0/3151549460 >> 172.21.15.129:0/3151549460 conn(0x7f57d80fcde0 msgr2=0x7f57d80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 -- 172.21.15.129:0/3151549460 shutdown_connections 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 -- 172.21.15.129:0/3151549460 wait complete. 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 Processor -- start 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 -- start start 2024-04-26T21:17:33.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8101870 0x7f57d8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8102a60 0x7f57d81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8103420 0x7f57d81a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810e520 con 0x7f57d8103420 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810e3a0 con 0x7f57d8102a60 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dfd9d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810e6a0 con 0x7f57d8101870 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57de33a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8103420 0x7f57d81a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57de33a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8103420 0x7f57d81a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40686/0 (socket says 172.21.15.129:40686) 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57de33a640 1 -- 172.21.15.129:0/1295471777 learned_addr learned my addr 172.21.15.129:0/1295471777 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57ddb39640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8101870 0x7f57d8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.112+0000 7f57dd338640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8102a60 0x7f57d81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57de33a640 1 -- 172.21.15.129:0/1295471777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8101870 msgr2=0x7f57d8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57de33a640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8101870 0x7f57d8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57de33a640 1 -- 172.21.15.129:0/1295471777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8102a60 msgr2=0x7f57d81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57de33a640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8102a60 0x7f57d81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57de33a640 1 -- 172.21.15.129:0/1295471777 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d81a0890 con 0x7f57d8103420 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57de33a640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8103420 0x7f57d81a01f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f57d000eaa0 tx=0x7f57d000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57ceffd640 1 -- 172.21.15.129:0/1295471777 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0004d20 con 0x7f57d8103420 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57ceffd640 1 -- 172.21.15.129:0/1295471777 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d0005020 con 0x7f57d8103420 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d81a0b80 con 0x7f57d8103420 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57ceffd640 1 -- 172.21.15.129:0/1295471777 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d00116b0 con 0x7f57d8103420 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.113+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d81a1080 con 0x7f57d8103420 2024-04-26T21:17:33.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.114+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d8108ca0 con 0x7f57d8103420 2024-04-26T21:17:33.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.115+0000 7f57ceffd640 1 -- 172.21.15.129:0/1295471777 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f57d00051d0 con 0x7f57d8103420 2024-04-26T21:17:33.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.116+0000 7f57ceffd640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c40a3c40 0x7f57c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.116+0000 7f57ddb39640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c40a3c40 0x7f57c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.116+0000 7f57ceffd640 1 -- 172.21.15.129:0/1295471777 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f57d00d0c10 con 0x7f57d8103420 2024-04-26T21:17:33.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.116+0000 7f57ddb39640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c40a3c40 0x7f57c40a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f57d819a150 tx=0x7f57d4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:33.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.120+0000 7f57ceffd640 1 -- 172.21.15.129:0/1295471777 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f57d0016030 con 0x7f57d8103420 2024-04-26T21:17:33.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.298+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f57d802dad0 con 0x7f57d8103420 2024-04-26T21:17:33.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.298+0000 7f57ceffd640 1 -- 172.21.15.129:0/1295471777 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f57d009d480 con 0x7f57d8103420 2024-04-26T21:17:33.299 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-26T21:17:33.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c40a3c40 msgr2=0x7f57c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c40a3c40 0x7f57c40a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f57d819a150 tx=0x7f57d4009110 comp rx=0 tx=0).stop 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8103420 msgr2=0x7f57d81a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8103420 0x7f57d81a01f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f57d000eaa0 tx=0x7f57d000eea0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 shutdown_connections 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c40a3c40 0x7f57c40a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8102a60 0x7f57d81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8103420 0x7f57d81a01f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 --2- 172.21.15.129:0/1295471777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8101870 0x7f57d8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 >> 172.21.15.129:0/1295471777 conn(0x7f57d80fcde0 msgr2=0x7f57d8105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 shutdown_connections 2024-04-26T21:17:33.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.300+0000 7f57dfd9d640 1 -- 172.21.15.129:0/1295471777 wait complete. 2024-04-26T21:17:33.312 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:17:33.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.414+0000 7fc141a28640 1 -- 172.21.15.129:0/4102790682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c075390 msgr2=0x7fc13c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.414+0000 7fc141a28640 1 --2- 172.21.15.129:0/4102790682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c075390 0x7fc13c075810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc124009900 tx=0x7fc12402a410 comp rx=0 tx=0).stop 2024-04-26T21:17:33.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.414+0000 7fc141a28640 1 -- 172.21.15.129:0/4102790682 shutdown_connections 2024-04-26T21:17:33.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.414+0000 7fc141a28640 1 --2- 172.21.15.129:0/4102790682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c10a840 0x7fc13c10ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.414+0000 7fc141a28640 1 --2- 172.21.15.129:0/4102790682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c075390 0x7fc13c075810 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.414+0000 7fc141a28640 1 --2- 172.21.15.129:0/4102790682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc13c076970 0x7fc13c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.414+0000 7fc141a28640 1 -- 172.21.15.129:0/4102790682 >> 172.21.15.129:0/4102790682 conn(0x7fc13c0fdf10 msgr2=0x7fc13c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:33.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.414+0000 7fc141a28640 1 -- 172.21.15.129:0/4102790682 shutdown_connections 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.414+0000 7fc141a28640 1 -- 172.21.15.129:0/4102790682 wait complete. 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc141a28640 1 Processor -- start 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc141a28640 1 -- start start 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc141a28640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c075390 0x7fc13c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc141a28640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc13c076970 0x7fc13c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc141a28640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c10a840 0x7fc13c1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc141a28640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1106e0 con 0x7fc13c10a840 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc141a28640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c110560 con 0x7fc13c076970 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc141a28640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c110860 con 0x7fc13c075390 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c10a840 0x7fc13c1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c10a840 0x7fc13c1a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40694/0 (socket says 172.21.15.129:40694) 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13b7fe640 1 -- 172.21.15.129:0/144722739 learned_addr learned my addr 172.21.15.129:0/144722739 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc133fff640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc13c076970 0x7fc13c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13affd640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c075390 0x7fc13c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13b7fe640 1 -- 172.21.15.129:0/144722739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c075390 msgr2=0x7fc13c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13b7fe640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c075390 0x7fc13c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13b7fe640 1 -- 172.21.15.129:0/144722739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc13c076970 msgr2=0x7fc13c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13b7fe640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc13c076970 0x7fc13c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13b7fe640 1 -- 172.21.15.129:0/144722739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c1a2a30 con 0x7fc13c10a840 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc13b7fe640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c10a840 0x7fc13c1a23c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc13400cad0 tx=0x7fc13400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc140a26640 1 -- 172.21.15.129:0/144722739 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13400ae20 con 0x7fc13c10a840 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc140a26640 1 -- 172.21.15.129:0/144722739 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc134005020 con 0x7fc13c10a840 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13c1a2d20 con 0x7fc13c10a840 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.415+0000 7fc140a26640 1 -- 172.21.15.129:0/144722739 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13400f6b0 con 0x7fc13c10a840 2024-04-26T21:17:33.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.416+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13c1a31e0 con 0x7fc13c10a840 2024-04-26T21:17:33.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.417+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc13c109570 con 0x7fc13c10a840 2024-04-26T21:17:33.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.417+0000 7fc140a26640 1 -- 172.21.15.129:0/144722739 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc1340051d0 con 0x7fc13c10a840 2024-04-26T21:17:33.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.418+0000 7fc140a26640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc1280a3c40 0x7fc1280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.418+0000 7fc13affd640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc1280a3c40 0x7fc1280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.418+0000 7fc140a26640 1 -- 172.21.15.129:0/144722739 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc1340d0520 con 0x7fc13c10a840 2024-04-26T21:17:33.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.419+0000 7fc13affd640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc1280a3c40 0x7fc1280a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0058c0 tx=0x7fc12c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:33.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.423+0000 7fc140a26640 1 -- 172.21.15.129:0/144722739 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc134014030 con 0x7fc13c10a840 2024-04-26T21:17:33.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.597+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc13c0635d0 con 0x7fc13c10a840 2024-04-26T21:17:33.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.597+0000 7fc140a26640 1 -- 172.21.15.129:0/144722739 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc13409c740 con 0x7fc13c10a840 2024-04-26T21:17:33.599 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-26T21:17:33.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc1280a3c40 msgr2=0x7fc1280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc1280a3c40 0x7fc1280a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0058c0 tx=0x7fc12c00a2d0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c10a840 msgr2=0x7fc13c1a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c10a840 0x7fc13c1a23c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc13400cad0 tx=0x7fc13400cf90 comp rx=0 tx=0).stop 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 shutdown_connections 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc1280a3c40 0x7fc1280a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc13c076970 0x7fc13c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc13c10a840 0x7fc13c1a23c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 --2- 172.21.15.129:0/144722739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c075390 0x7fc13c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 >> 172.21.15.129:0/144722739 conn(0x7fc13c0fdf10 msgr2=0x7fc13c106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 shutdown_connections 2024-04-26T21:17:33.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.599+0000 7fc141a28640 1 -- 172.21.15.129:0/144722739 wait complete. 2024-04-26T21:17:33.610 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:33 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3610530501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:33.611 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:33 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1903213247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:33.611 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:33 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1647460806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:33.611 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:33 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1295471777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:33.612 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:17:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:33 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3610530501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:33 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1903213247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:33 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1647460806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:33 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1295471777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:33.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.708+0000 7f38790bd640 1 -- 172.21.15.129:0/371647935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874101880 msgr2=0x7f3874101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.708+0000 7f38790bd640 1 --2- 172.21.15.129:0/371647935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874101880 0x7f3874101ca0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f386c009ca0 tx=0x7f386c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.708+0000 7f38790bd640 1 -- 172.21.15.129:0/371647935 shutdown_connections 2024-04-26T21:17:33.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.708+0000 7f38790bd640 1 --2- 172.21.15.129:0/371647935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874102a70 0x7f3874102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.708+0000 7f38790bd640 1 --2- 172.21.15.129:0/371647935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3874103430 0x7f3874109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.708+0000 7f38790bd640 1 --2- 172.21.15.129:0/371647935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874101880 0x7f3874101ca0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.708+0000 7f38790bd640 1 -- 172.21.15.129:0/371647935 >> 172.21.15.129:0/371647935 conn(0x7f38740fce10 msgr2=0x7f38740ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:33.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.708+0000 7f38790bd640 1 -- 172.21.15.129:0/371647935 shutdown_connections 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.708+0000 7f38790bd640 1 -- 172.21.15.129:0/371647935 wait complete. 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.709+0000 7f38790bd640 1 Processor -- start 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.709+0000 7f38790bd640 1 -- start start 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.709+0000 7f38790bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874101880 0x7f3874197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.709+0000 7f38790bd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874102a70 0x7f3874198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.709+0000 7f38790bd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3874103430 0x7f387419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.709+0000 7f38790bd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f387410d3b0 con 0x7f3874102a70 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.709+0000 7f38790bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f387410d230 con 0x7f3874101880 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.709+0000 7f38790bd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f387410d530 con 0x7f3874103430 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874102a70 0x7f3874198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874102a70 0x7f3874198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40718/0 (socket says 172.21.15.129:40718) 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387259c640 1 -- 172.21.15.129:0/338407186 learned_addr learned my addr 172.21.15.129:0/338407186 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f3872d9d640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874101880 0x7f3874197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387359e640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3874103430 0x7f387419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387259c640 1 -- 172.21.15.129:0/338407186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3874103430 msgr2=0x7f387419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387259c640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3874103430 0x7f387419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387259c640 1 -- 172.21.15.129:0/338407186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874101880 msgr2=0x7f3874197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387259c640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874101880 0x7f3874197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387259c640 1 -- 172.21.15.129:0/338407186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387419f700 con 0x7f3874102a70 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387359e640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3874103430 0x7f387419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f387259c640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874102a70 0x7f3874198020 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f386800d950 tx=0x7f386800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f385bfff640 1 -- 172.21.15.129:0/338407186 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868017030 con 0x7f3874102a70 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387419f9f0 con 0x7f3874102a70 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f385bfff640 1 -- 172.21.15.129:0/338407186 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3868004020 con 0x7f3874102a70 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f385bfff640 1 -- 172.21.15.129:0/338407186 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38680047a0 con 0x7f3874102a70 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.710+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387419ff90 con 0x7f3874102a70 2024-04-26T21:17:33.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.711+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3874108c70 con 0x7f3874102a70 2024-04-26T21:17:33.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.713+0000 7f385bfff640 1 -- 172.21.15.129:0/338407186 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3868009a30 con 0x7f3874102a70 2024-04-26T21:17:33.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.713+0000 7f385bfff640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f385c0a3c40 0x7f385c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:33.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.714+0000 7f3872d9d640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f385c0a3c40 0x7f385c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:33.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.714+0000 7f385bfff640 1 -- 172.21.15.129:0/338407186 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f38680d0730 con 0x7f3874102a70 2024-04-26T21:17:33.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.714+0000 7f3872d9d640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f385c0a3c40 0x7f385c0a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f386c000f80 tx=0x7f386c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:33.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.717+0000 7f385bfff640 1 -- 172.21.15.129:0/338407186 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f386809cf40 con 0x7f3874102a70 2024-04-26T21:17:33.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:33 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3610530501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:33.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:33 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1903213247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:33.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:33 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1647460806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:33.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:33 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1295471777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:33.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.932+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f387402dc80 con 0x7f3874102a70 2024-04-26T21:17:33.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.932+0000 7f385bfff640 1 -- 172.21.15.129:0/338407186 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f386809c840 con 0x7f3874102a70 2024-04-26T21:17:33.933 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:33.934 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:17:33.934 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f385c0a3c40 msgr2=0x7f385c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f385c0a3c40 0x7f385c0a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f386c000f80 tx=0x7f386c005dc0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874102a70 msgr2=0x7f3874198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874102a70 0x7f3874198020 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f386800d950 tx=0x7f386800de10 comp rx=0 tx=0).stop 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 shutdown_connections 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f385c0a3c40 0x7f385c0a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3874103430 0x7f387419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3874102a70 0x7f3874198020 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 --2- 172.21.15.129:0/338407186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3874101880 0x7f3874197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.934+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 >> 172.21.15.129:0/338407186 conn(0x7f38740fce10 msgr2=0x7f3874105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:33.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.935+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 shutdown_connections 2024-04-26T21:17:33.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:33.935+0000 7f38790bd640 1 -- 172.21.15.129:0/338407186 wait complete. 2024-04-26T21:17:33.947 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:17:34.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.044+0000 7fbaaa683640 1 -- 172.21.15.129:0/1943313023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 msgr2=0x7fbaa4101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.044+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1943313023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 0x7fbaa4101c90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fba98009900 tx=0x7fba9802a410 comp rx=0 tx=0).stop 2024-04-26T21:17:34.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.044+0000 7fbaaa683640 1 -- 172.21.15.129:0/1943313023 shutdown_connections 2024-04-26T21:17:34.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.044+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1943313023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbaa4103420 0x7fbaa4109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.044+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1943313023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa4102a60 0x7fbaa4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.044+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1943313023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 0x7fbaa4101c90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.044+0000 7fbaaa683640 1 -- 172.21.15.129:0/1943313023 >> 172.21.15.129:0/1943313023 conn(0x7fbaa40fcde0 msgr2=0x7fbaa40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.044+0000 7fbaaa683640 1 -- 172.21.15.129:0/1943313023 shutdown_connections 2024-04-26T21:17:34.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaaa683640 1 -- 172.21.15.129:0/1943313023 wait complete. 2024-04-26T21:17:34.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaaa683640 1 Processor -- start 2024-04-26T21:17:34.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaaa683640 1 -- start start 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaaa683640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 0x7fbaa4105800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaaa683640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa4102a60 0x7fbaa4103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaaa683640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbaa4103420 0x7fbaa4104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaaa683640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa410d390 con 0x7fbaa4101870 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaaa683640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa410d210 con 0x7fbaa4103420 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaaa683640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa410d510 con 0x7fbaa4102a60 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaa3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 0x7fbaa4105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaa3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 0x7fbaa4105800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40730/0 (socket says 172.21.15.129:40730) 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.045+0000 7fbaa3fff640 1 -- 172.21.15.129:0/1150276398 learned_addr learned my addr 172.21.15.129:0/1150276398 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa37fe640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa4102a60 0x7fbaa4103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa8c20640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbaa4103420 0x7fbaa4104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa3fff640 1 -- 172.21.15.129:0/1150276398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa4102a60 msgr2=0x7fbaa4103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa3fff640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa4102a60 0x7fbaa4103e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa3fff640 1 -- 172.21.15.129:0/1150276398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbaa4103420 msgr2=0x7fbaa4104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa3fff640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbaa4103420 0x7fbaa4104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa3fff640 1 -- 172.21.15.129:0/1150276398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa4104c10 con 0x7fbaa4101870 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa8c20640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbaa4103420 0x7fbaa4104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa3fff640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 0x7fbaa4105800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fba98006010 tx=0x7fba980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa17fa640 1 -- 172.21.15.129:0/1150276398 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9803f030 con 0x7fbaa4101870 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa41a2cd0 con 0x7fbaa4101870 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa17fa640 1 -- 172.21.15.129:0/1150276398 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba980318f0 con 0x7fbaa4101870 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaa17fa640 1 -- 172.21.15.129:0/1150276398 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98035070 con 0x7fbaa4101870 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.046+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa41a31c0 con 0x7fbaa4101870 2024-04-26T21:17:34.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.047+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa41a3390 con 0x7fbaa4101870 2024-04-26T21:17:34.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.048+0000 7fbaa17fa640 1 -- 172.21.15.129:0/1150276398 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fba98042030 con 0x7fbaa4101870 2024-04-26T21:17:34.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.049+0000 7fbaa17fa640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fba8c0a3c40 0x7fba8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.049+0000 7fbaa37fe640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fba8c0a3c40 0x7fba8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.049+0000 7fbaa17fa640 1 -- 172.21.15.129:0/1150276398 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fba980f0b00 con 0x7fbaa4101870 2024-04-26T21:17:34.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.049+0000 7fbaa37fe640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fba8c0a3c40 0x7fba8c0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4105570 tx=0x7fba9400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.054+0000 7fbaa17fa640 1 -- 172.21.15.129:0/1150276398 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fba980bd370 con 0x7fbaa4101870 2024-04-26T21:17:34.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.231+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbaa41a3720 con 0x7fbaa4101870 2024-04-26T21:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.232+0000 7fbaa17fa640 1 -- 172.21.15.129:0/1150276398 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7fba980bcc70 con 0x7fbaa4101870 2024-04-26T21:17:34.235 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:34.235 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.234+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fba8c0a3c40 msgr2=0x7fba8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.234+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fba8c0a3c40 0x7fba8c0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4105570 tx=0x7fba9400b390 comp rx=0 tx=0).stop 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.234+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 msgr2=0x7fbaa4105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.234+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 0x7fbaa4105800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fba98006010 tx=0x7fba980042f0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.235+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 shutdown_connections 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.235+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fba8c0a3c40 0x7fba8c0a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.235+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbaa4103420 0x7fbaa4104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.235+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa4102a60 0x7fbaa4103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.235+0000 7fbaaa683640 1 --2- 172.21.15.129:0/1150276398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa4101870 0x7fbaa4105800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.235+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 >> 172.21.15.129:0/1150276398 conn(0x7fbaa40fcde0 msgr2=0x7fbaa4069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.235+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 shutdown_connections 2024-04-26T21:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.235+0000 7fbaaa683640 1 -- 172.21.15.129:0/1150276398 wait complete. 2024-04-26T21:17:34.248 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:17:34.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.344+0000 7f0d93ada640 1 -- 172.21.15.129:0/2718501384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c103400 msgr2=0x7f0d8c1057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.344+0000 7f0d93ada640 1 --2- 172.21.15.129:0/2718501384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c103400 0x7f0d8c1057f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009900 tx=0x7f0d8802a410 comp rx=0 tx=0).stop 2024-04-26T21:17:34.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 -- 172.21.15.129:0/2718501384 shutdown_connections 2024-04-26T21:17:34.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 --2- 172.21.15.129:0/2718501384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c105d30 0x7f0d8c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 --2- 172.21.15.129:0/2718501384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d8c108660 0x7f0d8c10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 --2- 172.21.15.129:0/2718501384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c103400 0x7f0d8c1057f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 -- 172.21.15.129:0/2718501384 >> 172.21.15.129:0/2718501384 conn(0x7f0d8c0fce10 msgr2=0x7f0d8c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 -- 172.21.15.129:0/2718501384 shutdown_connections 2024-04-26T21:17:34.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 -- 172.21.15.129:0/2718501384 wait complete. 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 Processor -- start 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 -- start start 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c103400 0x7f0d8c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d8c105d30 0x7f0d8c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c108660 0x7f0d8c1a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c10e4b0 con 0x7f0d8c108660 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c10e330 con 0x7f0d8c103400 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d93ada640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c10e630 con 0x7f0d8c105d30 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.345+0000 7f0d92077640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c108660 0x7f0d8c1a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d92077640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c108660 0x7f0d8c1a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40762/0 (socket says 172.21.15.129:40762) 2024-04-26T21:17:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d92077640 1 -- 172.21.15.129:0/263157925 learned_addr learned my addr 172.21.15.129:0/263157925 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d91876640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c103400 0x7f0d8c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d91075640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d8c105d30 0x7f0d8c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d92077640 1 -- 172.21.15.129:0/263157925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d8c105d30 msgr2=0x7f0d8c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d92077640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d8c105d30 0x7f0d8c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d92077640 1 -- 172.21.15.129:0/263157925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c103400 msgr2=0x7f0d8c19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d92077640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c103400 0x7f0d8c19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d92077640 1 -- 172.21.15.129:0/263157925 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c1a2a90 con 0x7f0d8c108660 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d92077640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c108660 0x7f0d8c1a2420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009fd0 tx=0x7f0d8400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d82ffd640 1 -- 172.21.15.129:0/263157925 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84017070 con 0x7f0d8c108660 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d82ffd640 1 -- 172.21.15.129:0/263157925 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d84003e20 con 0x7f0d8c108660 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c1a2d80 con 0x7f0d8c108660 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d82ffd640 1 -- 172.21.15.129:0/263157925 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84010460 con 0x7f0d8c108660 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.346+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c1a3270 con 0x7f0d8c108660 2024-04-26T21:17:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.347+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8c10a1b0 con 0x7f0d8c108660 2024-04-26T21:17:34.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.348+0000 7f0d82ffd640 1 -- 172.21.15.129:0/263157925 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0d84003970 con 0x7f0d8c108660 2024-04-26T21:17:34.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.349+0000 7f0d82ffd640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0d780a3cc0 0x7f0d780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.349+0000 7f0d91876640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0d780a3cc0 0x7f0d780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.349+0000 7f0d82ffd640 1 -- 172.21.15.129:0/263157925 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0d840d02e0 con 0x7f0d8c108660 2024-04-26T21:17:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.349+0000 7f0d91876640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0d780a3cc0 0x7f0d780a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0d88000c00 tx=0x7f0d880314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.353+0000 7f0d82ffd640 1 -- 172.21.15.129:0/263157925 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0d8409cb50 con 0x7f0d8c108660 2024-04-26T21:17:34.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.566+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d8c02dad0 con 0x7f0d8c108660 2024-04-26T21:17:34.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.567+0000 7f0d82ffd640 1 -- 172.21.15.129:0/263157925 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f0d8409c450 con 0x7f0d8c108660 2024-04-26T21:17:34.569 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:34.569 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:17:34.570 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:17:34.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0d780a3cc0 msgr2=0x7f0d780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0d780a3cc0 0x7f0d780a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0d88000c00 tx=0x7f0d880314a0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c108660 msgr2=0x7f0d8c1a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c108660 0x7f0d8c1a2420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009fd0 tx=0x7f0d8400b260 comp rx=0 tx=0).stop 2024-04-26T21:17:34.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 shutdown_connections 2024-04-26T21:17:34.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0d780a3cc0 0x7f0d780a6180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d8c108660 0x7f0d8c1a2420 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d8c105d30 0x7f0d8c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 --2- 172.21.15.129:0/263157925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c103400 0x7f0d8c19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 >> 172.21.15.129:0/263157925 conn(0x7f0d8c0fce10 msgr2=0x7f0d8c102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 shutdown_connections 2024-04-26T21:17:34.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.569+0000 7f0d93ada640 1 -- 172.21.15.129:0/263157925 wait complete. 2024-04-26T21:17:34.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:34 smithi129 ceph-mon[29237]: pgmap v337: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:17:34.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:34 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/144722739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:34.582 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:34 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/338407186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:34.582 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:34 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1150276398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:34.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:16:03.677644+0000', 'modified': '2024-04-26T21:16:08.015827+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24497}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6839/2157967831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 2157967831}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 2157967831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/3984403456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 3984403456}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 3984403456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24497]} 2024-04-26T21:17:34.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:17:34.583 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:17:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:34 smithi150 ceph-mon[32290]: pgmap v337: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:17:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:34 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/144722739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:34 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/338407186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:34 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1150276398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:34.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.680+0000 7f35cbb96640 1 -- 172.21.15.129:0/3919056928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 msgr2=0x7f35c4108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.680+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3919056928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 0x7f35c4108110 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009900 tx=0x7f35b802a410 comp rx=0 tx=0).stop 2024-04-26T21:17:34.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.680+0000 7f35cbb96640 1 -- 172.21.15.129:0/3919056928 shutdown_connections 2024-04-26T21:17:34.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.680+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3919056928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35c4108650 0x7f35c410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.680+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3919056928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 0x7f35c4108110 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.680+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3919056928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c41033f0 0x7f35c41057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.680+0000 7f35cbb96640 1 -- 172.21.15.129:0/3919056928 >> 172.21.15.129:0/3919056928 conn(0x7f35c40fcde0 msgr2=0x7f35c40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.680+0000 7f35cbb96640 1 -- 172.21.15.129:0/3919056928 shutdown_connections 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.680+0000 7f35cbb96640 1 -- 172.21.15.129:0/3919056928 wait complete. 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35cbb96640 1 Processor -- start 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35cbb96640 1 -- start start 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35cbb96640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c41033f0 0x7f35c4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35cbb96640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 0x7f35c41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35cbb96640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35c4108650 0x7f35c41a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35cbb96640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410e4a0 con 0x7f35c4105d20 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35cbb96640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410e320 con 0x7f35c41033f0 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35cbb96640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410e620 con 0x7f35c4108650 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35c9131640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 0x7f35c41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35c9131640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 0x7f35c41991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40780/0 (socket says 172.21.15.129:40780) 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35c9131640 1 -- 172.21.15.129:0/3109636439 learned_addr learned my addr 172.21.15.129:0/3109636439 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35c9932640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c41033f0 0x7f35c4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.681+0000 7f35ca133640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35c4108650 0x7f35c41a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35c9131640 1 -- 172.21.15.129:0/3109636439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35c4108650 msgr2=0x7f35c41a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35c9131640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35c4108650 0x7f35c41a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35c9131640 1 -- 172.21.15.129:0/3109636439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c41033f0 msgr2=0x7f35c4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35c9131640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c41033f0 0x7f35c4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35c9131640 1 -- 172.21.15.129:0/3109636439 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c41a08d0 con 0x7f35c4105d20 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35ca133640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35c4108650 0x7f35c41a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35c9131640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 0x7f35c41991d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006010 tx=0x7f35b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35c2ffd640 1 -- 172.21.15.129:0/3109636439 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b803f030 con 0x7f35c4105d20 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35cbb96640 1 -- 172.21.15.129:0/3109636439 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c41a0b60 con 0x7f35c4105d20 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35c2ffd640 1 -- 172.21.15.129:0/3109636439 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b8003cb0 con 0x7f35c4105d20 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35c2ffd640 1 -- 172.21.15.129:0/3109636439 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8035070 con 0x7f35c4105d20 2024-04-26T21:17:34.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.682+0000 7f35cbb96640 1 -- 172.21.15.129:0/3109636439 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c41a1050 con 0x7f35c4105d20 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.684+0000 7f35c2ffd640 1 -- 172.21.15.129:0/3109636439 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f35b8042030 con 0x7f35c4105d20 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.684+0000 7f35ce988740 1 Processor -- start 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ce988740 1 -- start start 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ce988740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a6dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ce988740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6e080 0x556468a72b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ce988740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a731f0 0x556468a75680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ce988740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564689d2740 con 0x556468a6d690 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ce988740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55646896e3a0 con 0x556468a731f0 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ce988740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5564689ce700 con 0x556468a6e080 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35c2ffd640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f35ac0a3c40 0x7f35ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ca133640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a6dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ca133640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a6dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40782/0 (socket says 172.21.15.129:40782) 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ca133640 1 -- 172.21.15.129:0/628776265 learned_addr learned my addr 172.21.15.129:0/628776265 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ca133640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f35ac0a3c40 0x7f35ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35c9131640 1 --2- 172.21.15.129:0/628776265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a731f0 0x556468a75680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35c9932640 1 --2- 172.21.15.129:0/628776265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6e080 0x556468a72b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35c2ffd640 1 -- 172.21.15.129:0/3109636439 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f35b80f0be0 con 0x7f35c4105d20 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ca133640 1 -- 172.21.15.129:0/628776265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6e080 msgr2=0x556468a72b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ca133640 1 --2- 172.21.15.129:0/628776265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6e080 0x556468a72b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ca133640 1 -- 172.21.15.129:0/628776265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a731f0 msgr2=0x556468a75680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ca133640 1 --2- 172.21.15.129:0/628776265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a731f0 0x556468a75680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.685+0000 7f35ca133640 1 -- 172.21.15.129:0/628776265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646898a750 con 0x556468a6d690 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35ca133640 1 --2- 172.21.15.129:0/628776265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a6dab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f35b4013e40 tx=0x7f35b40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd02e61dbeaf421 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35ca133640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f35ac0a3c40 0x7f35ac0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f35b4010ca0 tx=0x7f35b401b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35abfff640 1 -- 172.21.15.129:0/628776265 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b400bd50 con 0x556468a6d690 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35abfff640 1 -- 172.21.15.129:0/628776265 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b4003e20 con 0x556468a6d690 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35abfff640 1 -- 172.21.15.129:0/628776265 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b40046e0 con 0x556468a6d690 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35ce988740 1 -- 172.21.15.129:0/628776265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 msgr2=0x556468a6dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35ce988740 1 --2- 172.21.15.129:0/628776265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a6dab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f35b4013e40 tx=0x7f35b40107a0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35ce988740 1 -- 172.21.15.129:0/628776265 shutdown_connections 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35ce988740 1 --2- 172.21.15.129:0/628776265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a731f0 0x556468a75680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35ce988740 1 --2- 172.21.15.129:0/628776265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6e080 0x556468a72b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35ce988740 1 --2- 172.21.15.129:0/628776265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a6dab0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.686+0000 7f35ce988740 1 -- 172.21.15.129:0/628776265 >> 172.21.15.129:0/628776265 conn(0x556468a68d00 msgr2=0x556468a69100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 -- 172.21.15.129:0/628776265 shutdown_connections 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 -- 172.21.15.129:0/628776265 wait complete. 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 -- start start 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 Processor -- start 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a77b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a731f0 0x556468a7f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556468a76140 con 0x556468a6d690 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556468959fe0 con 0x556468a6e080 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556468a762c0 con 0x556468a731f0 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ce988740 10 client.0 _ll_register_callbacks cb 0x556468970150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:17:34.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35c9932640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a77b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35c9932640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a77b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40786/0 (socket says 172.21.15.129:40786) 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35c9932640 1 -- 172.21.15.129:0/3753503718 learned_addr learned my addr 172.21.15.129:0/3753503718 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.687+0000 7f35ca133640 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a731f0 0x556468a7f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35c9131640 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35c9932640 1 -- 172.21.15.129:0/3753503718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a731f0 msgr2=0x556468a7f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35c9932640 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a731f0 0x556468a7f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35c9932640 1 -- 172.21.15.129:0/3753503718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 msgr2=0x556468a780b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35c9932640 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a780b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35c9932640 1 -- 172.21.15.129:0/3753503718 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564686e42d0 con 0x556468a6d690 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35c9932640 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a77b70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f35bc005fd0 tx=0x7f35bc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdac0397d993a847 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35a97fa640 1 -- 172.21.15.129:0/3753503718 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc004020 con 0x556468a6d690 2024-04-26T21:17:34.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35a97fa640 1 -- 172.21.15.129:0/3753503718 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f35bc0041c0 con 0x556468a6d690 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35ce988740 1 -- 172.21.15.129:0/3753503718 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b4003e20 con 0x556468a6d690 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35a97fa640 1 -- 172.21.15.129:0/3753503718 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f35bc004400 con 0x556468a6d690 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35ce988740 10 client.15093 fetch_fsmap learned FSMap version 19 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35ce988740 1 -- 172.21.15.129:0/3753503718 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5564687ae6b0 con 0x556468a6d690 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35a97fa640 1 -- 172.21.15.129:0/3753503718 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b4003e20 con 0x556468a6d690 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35a97fa640 1 -- 172.21.15.129:0/3753503718 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f35bc004ca0 con 0x556468a6d690 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35ce988740 10 client.15093 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.688+0000 7f35ce988740 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.689+0000 7f35ce988740 20 client.15093 populate_metadata read hostname 'smithi129' 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.689+0000 7f35ce988740 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556468a84600 0x556468a85a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.689+0000 7f35ce988740 4 client.15093 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.689+0000 7f35ce988740 1 -- 172.21.15.129:0/3753503718 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564689e0350 con 0x556468a84600 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.689+0000 7f35c9131640 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556468a84600 0x556468a85a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.689+0000 7f35a97fa640 10 client.15093 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.689+0000 7f35c9131640 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556468a84600 0x556468a85a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4510532ad1481739 server_cookie=c9c647f4ca2649a2 in_seq=0 out_seq=0 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35a97fa640 1 -- 172.21.15.129:0/3753503718 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564689e0350 con 0x556468a84600 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35a97fa640 10 client.15093 handle_command_reply: tid=0 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 client.15093 shutdown 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 -- 172.21.15.129:0/3753503718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 msgr2=0x556468a77b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a77b70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f35bc005fd0 tx=0x7f35bc0118d0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 -- 172.21.15.129:0/3753503718 shutdown_connections 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556468a84600 0x556468a85a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a780b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a731f0 0x556468a7f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35a97fa640 0 client.15093 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 --2- 172.21.15.129:0/3753503718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a77b70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 -- 172.21.15.129:0/3753503718 >> 172.21.15.129:0/3753503718 conn(0x556468a68d00 msgr2=0x556468a67f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 -- 172.21.15.129:0/3753503718 shutdown_connections 2024-04-26T21:17:34.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 1 -- 172.21.15.129:0/3753503718 wait complete. 2024-04-26T21:17:34.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.690+0000 7f35ce988740 20 client.15093 trim_cache size 0 max 16384 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ce988740 1 Processor -- start 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ce988740 1 -- start start 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ce988740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a68d00 0x556468a690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ce988740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a7f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ce988740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a6c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ce988740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556468a83db0 con 0x556468a6d690 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ce988740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556468a84480 con 0x556468a6e080 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ce988740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556468a84b50 con 0x556468a68d00 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9131640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a7f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9131640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a7f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40794/0 (socket says 172.21.15.129:40794) 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9131640 1 -- 172.21.15.129:0/2528068983 learned_addr learned my addr 172.21.15.129:0/2528068983 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9932640 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a68d00 0x556468a690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ca133640 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a6c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9131640 1 -- 172.21.15.129:0/2528068983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a68d00 msgr2=0x556468a690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9131640 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a68d00 0x556468a690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9131640 1 -- 172.21.15.129:0/2528068983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 msgr2=0x556468a6c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9131640 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a6c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9131640 1 -- 172.21.15.129:0/2528068983 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556468a76c30 con 0x556468a6d690 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ca133640 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a6c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35c9131640 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a7f010 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f35b80b9720 tx=0x7f35b80dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbf24bf9511ad46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ab7fe640 1 -- 172.21.15.129:0/2528068983 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8049040 con 0x556468a6d690 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ab7fe640 1 -- 172.21.15.129:0/2528068983 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b80d62a0 con 0x556468a6d690 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.709+0000 7f35ab7fe640 1 -- 172.21.15.129:0/2528068983 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b80c59c0 con 0x556468a6d690 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 -- 172.21.15.129:0/2528068983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 msgr2=0x556468a7f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a7f010 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f35b80b9720 tx=0x7f35b80dd040 comp rx=0 tx=0).stop 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 -- 172.21.15.129:0/2528068983 shutdown_connections 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a6c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a6d690 0x556468a7f010 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 --2- 172.21.15.129:0/2528068983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a68d00 0x556468a690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 -- 172.21.15.129:0/2528068983 >> 172.21.15.129:0/2528068983 conn(0x556468a6ee60 msgr2=0x556468a6bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 -- 172.21.15.129:0/2528068983 shutdown_connections 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 -- 172.21.15.129:0/2528068983 wait complete. 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 -- start start 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.710+0000 7f35ce988740 1 Processor -- start 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ce988740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a68d00 0x556468a7aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ce988740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6d690 0x556468a7b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ce988740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a7e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ce988740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556468a858f0 con 0x556468a68d00 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ce988740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556468a85220 con 0x556468a6e080 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ce988740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556468a85fc0 con 0x556468a6d690 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ce988740 10 client.0 _ll_register_callbacks cb 0x7f35bc00bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9932640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a68d00 0x556468a7aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9932640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a68d00 0x556468a7aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40796/0 (socket says 172.21.15.129:40796) 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9932640 1 -- 172.21.15.129:0/208102217 learned_addr learned my addr 172.21.15.129:0/208102217 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ca133640 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a7e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9131640 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6d690 0x556468a7b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9932640 1 -- 172.21.15.129:0/208102217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6d690 msgr2=0x556468a7b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9932640 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6d690 0x556468a7b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9932640 1 -- 172.21.15.129:0/208102217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 msgr2=0x556468a7e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9932640 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a7e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9932640 1 -- 172.21.15.129:0/208102217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556468a7ea40 con 0x556468a68d00 2024-04-26T21:17:34.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ca133640 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a7e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35c9932640 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a68d00 0x556468a7aef0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f35bc000c00 tx=0x7f35bc0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b32878c18159d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35a97fa640 1 -- 172.21.15.129:0/208102217 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00b480 con 0x556468a68d00 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35a97fa640 1 -- 172.21.15.129:0/208102217 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f35bc010e40 con 0x556468a68d00 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.711+0000 7f35ce988740 1 -- 172.21.15.129:0/208102217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b80d62a0 con 0x556468a68d00 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35a97fa640 1 -- 172.21.15.129:0/208102217 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f35bc004660 con 0x556468a68d00 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35ce988740 10 client.15099 fetch_fsmap learned FSMap version 19 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35ce988740 1 -- 172.21.15.129:0/208102217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556468aeabf0 con 0x556468a68d00 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35a97fa640 1 -- 172.21.15.129:0/208102217 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b80d62a0 con 0x556468a68d00 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35a97fa640 1 -- 172.21.15.129:0/208102217 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f35bc030030 con 0x556468a68d00 2024-04-26T21:17:34.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35ce988740 10 client.15099 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35ce988740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35ce988740 20 client.15099 populate_metadata read hostname 'smithi129' 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35ce988740 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556468a731f0 0x556468aee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35ce988740 4 client.15099 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35ce988740 1 -- 172.21.15.129:0/208102217 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "status"}) -- 0x5564689da1c0 con 0x556468a731f0 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.712+0000 7f35c9131640 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556468a731f0 0x556468aee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35a97fa640 10 client.15099 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35c9131640 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556468a731f0 0x556468aee5b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b29e80e0c04bd9 server_cookie=ec651c9712df4d44 in_seq=0 out_seq=0 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35a97fa640 1 -- 172.21.15.129:0/208102217 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5564689da1c0 con 0x556468a731f0 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35a97fa640 10 client.15099 handle_command_reply: tid=0 2024-04-26T21:17:34.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 client.15099 shutdown 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 -- 172.21.15.129:0/208102217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a68d00 msgr2=0x556468a7aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a68d00 0x556468a7aef0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f35bc000c00 tx=0x7f35bc0052f0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 -- 172.21.15.129:0/208102217 shutdown_connections 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556468a731f0 0x556468aee5b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556468a6e080 0x556468a7e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35a97fa640 0 client.15099 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556468a6d690 0x556468a7b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 --2- 172.21.15.129:0/208102217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556468a68d00 0x556468a7aef0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 -- 172.21.15.129:0/208102217 >> 172.21.15.129:0/208102217 conn(0x556468a6ee60 msgr2=0x556468a67670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 -- 172.21.15.129:0/208102217 shutdown_connections 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 1 -- 172.21.15.129:0/208102217 wait complete. 2024-04-26T21:17:34.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.713+0000 7f35ce988740 20 client.15099 trim_cache size 0 max 16384 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "eccca2b2-0410-11ef-bc93-c7b262605968", 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 0, 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24497, 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 90.037372313000006, 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 19, 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 88, 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 86, 2024-04-26T21:17:34.723 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 95.782556935000002 2024-04-26T21:17:34.724 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:17:34.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 -- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f35ac0a3c40 msgr2=0x7f35ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f35ac0a3c40 0x7f35ac0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f35b4010ca0 tx=0x7f35b401b000 comp rx=0 tx=0).stop 2024-04-26T21:17:34.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 -- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 msgr2=0x7f35c41991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 0x7f35c41991d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006010 tx=0x7f35b80042f0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 -- 172.21.15.129:0/3109636439 shutdown_connections 2024-04-26T21:17:34.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f35ac0a3c40 0x7f35ac0a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35c4108650 0x7f35c41a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35c4105d20 0x7f35c41991d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 --2- 172.21.15.129:0/3109636439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c41033f0 0x7f35c4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 -- 172.21.15.129:0/3109636439 >> 172.21.15.129:0/3109636439 conn(0x7f35c40fcde0 msgr2=0x7f35c4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 -- 172.21.15.129:0/3109636439 shutdown_connections 2024-04-26T21:17:34.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:34.714+0000 7f35cbb96640 1 -- 172.21.15.129:0/3109636439 wait complete. 2024-04-26T21:17:34.727 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T21:17:34.727 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:17:34.739 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi129.front.sepia.ceph.com 2024-04-26T21:17:34.739 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T21:17:34.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:34 smithi171 ceph-mon[33478]: pgmap v337: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:17:34.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:34 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/144722739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:34.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:34 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/338407186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:34.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:34 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1150276398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:34.974 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:35.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:35 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/263157925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:35.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:35 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/263157925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:35.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:35 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/263157925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:36 smithi150 ceph-mon[32290]: pgmap v338: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:17:36.640 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:36 smithi129 ceph-mon[29237]: pgmap v338: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:17:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:36 smithi171 ceph-mon[33478]: pgmap v338: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:17:36.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.740+0000 7f148bebc700 1 -- 172.21.15.129:0/3291433083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 msgr2=0x7f148410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:36.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.740+0000 7f148bebc700 1 --2- 172.21.15.129:0/3291433083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 0x7f148410a7f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f147c00b0d0 tx=0x7f147c00b3e0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 -- 172.21.15.129:0/3291433083 shutdown_connections 2024-04-26T21:17:36.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 --2- 172.21.15.129:0/3291433083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 0x7f148410a7f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 --2- 172.21.15.129:0/3291433083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14841023a0 0x7f14841027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 --2- 172.21.15.129:0/3291433083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14841035a0 0x7f1484103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 -- 172.21.15.129:0/3291433083 >> 172.21.15.129:0/3291433083 conn(0x7f14840fd980 msgr2=0x7f14840ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:36.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 -- 172.21.15.129:0/3291433083 shutdown_connections 2024-04-26T21:17:36.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 -- 172.21.15.129:0/3291433083 wait complete. 2024-04-26T21:17:36.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 Processor -- start 2024-04-26T21:17:36.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 -- start start 2024-04-26T21:17:36.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14841023a0 0x7f148406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:36.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14841035a0 0x7f148419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:36.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 0x7f14841a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:36.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f148410de20 con 0x7f1484103f40 2024-04-26T21:17:36.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f148410dca0 con 0x7f14841023a0 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.741+0000 7f148bebc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f148410dfa0 con 0x7f14841035a0 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148a459700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 0x7f14841a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148a459700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 0x7f14841a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40820/0 (socket says 172.21.15.129:40820) 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148a459700 1 -- 172.21.15.129:0/2787836982 learned_addr learned my addr 172.21.15.129:0/2787836982 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f1489c58700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14841023a0 0x7f148406c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f1489457700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14841035a0 0x7f148419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148a459700 1 -- 172.21.15.129:0/2787836982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14841035a0 msgr2=0x7f148419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148a459700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14841035a0 0x7f148419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148a459700 1 -- 172.21.15.129:0/2787836982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14841023a0 msgr2=0x7f148406c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148a459700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14841023a0 0x7f148406c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148a459700 1 -- 172.21.15.129:0/2787836982 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14841a1190 con 0x7f1484103f40 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148a459700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 0x7f14841a08c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f147c009040 tx=0x7f147c004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f147affd700 1 -- 172.21.15.129:0/2787836982 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c01f030 con 0x7f1484103f40 2024-04-26T21:17:36.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14841a1420 con 0x7f1484103f40 2024-04-26T21:17:36.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f147affd700 1 -- 172.21.15.129:0/2787836982 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147c003a00 con 0x7f1484103f40 2024-04-26T21:17:36.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f147affd700 1 -- 172.21.15.129:0/2787836982 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0046e0 con 0x7f1484103f40 2024-04-26T21:17:36.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.742+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14841a18d0 con 0x7f1484103f40 2024-04-26T21:17:36.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.743+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148406a110 con 0x7f1484103f40 2024-04-26T21:17:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.744+0000 7f147affd700 1 -- 172.21.15.129:0/2787836982 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f147c003bb0 con 0x7f1484103f40 2024-04-26T21:17:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.745+0000 7f147affd700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f14700a3cd0 0x7f14700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.745+0000 7f1489c58700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f14700a3cd0 0x7f14700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.745+0000 7f147affd700 1 -- 172.21.15.129:0/2787836982 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f147c00f020 con 0x7f1484103f40 2024-04-26T21:17:36.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.745+0000 7f1489c58700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f14700a3cd0 0x7f14700a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1484103400 tx=0x7f147400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:36.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.749+0000 7f147affd700 1 -- 172.21.15.129:0/2787836982 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f147c029020 con 0x7f1484103f40 2024-04-26T21:17:36.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.965+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1484063860 con 0x7f14700a3cd0 2024-04-26T21:17:36.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.966+0000 7f147affd700 1 -- 172.21.15.129:0/2787836982 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1484063860 con 0x7f14700a3cd0 2024-04-26T21:17:36.968 INFO:teuthology.orchestra.run.smithi129.stdout:Backend: cephadm 2024-04-26T21:17:36.968 INFO:teuthology.orchestra.run.smithi129.stdout:Available: Yes 2024-04-26T21:17:36.968 INFO:teuthology.orchestra.run.smithi129.stdout:Paused: No 2024-04-26T21:17:36.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f14700a3cd0 msgr2=0x7f14700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:36.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f14700a3cd0 0x7f14700a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1484103400 tx=0x7f147400a380 comp rx=0 tx=0).stop 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 msgr2=0x7f14841a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 0x7f14841a08c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f147c009040 tx=0x7f147c004d50 comp rx=0 tx=0).stop 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 shutdown_connections 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f14700a3cd0 0x7f14700a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1484103f40 0x7f14841a08c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14841023a0 0x7f148406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 --2- 172.21.15.129:0/2787836982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14841035a0 0x7f148419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 >> 172.21.15.129:0/2787836982 conn(0x7f14840fd980 msgr2=0x7f1484106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 shutdown_connections 2024-04-26T21:17:36.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:36.968+0000 7f148bebc700 1 -- 172.21.15.129:0/2787836982 wait complete. 2024-04-26T21:17:37.797 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T21:17:38.041 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:38.414 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:38 smithi129 ceph-mon[29237]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:38.414 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:38 smithi129 ceph-mon[29237]: pgmap v339: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:17:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:38 smithi150 ceph-mon[32290]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:38 smithi150 ceph-mon[32290]: pgmap v339: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:17:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:38 smithi171 ceph-mon[33478]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:38 smithi171 ceph-mon[33478]: pgmap v339: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:17:39.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.766+0000 7fbfec92a700 1 -- 172.21.15.129:0/1048699281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8103570 msgr2=0x7fbfe81039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.766+0000 7fbfec92a700 1 --2- 172.21.15.129:0/1048699281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8103570 0x7fbfe81039d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80099c0 tx=0x7fbfd8009cd0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.766+0000 7fbfec92a700 1 -- 172.21.15.129:0/1048699281 shutdown_connections 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.766+0000 7fbfec92a700 1 --2- 172.21.15.129:0/1048699281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbfe8103f10 0x7fbfe810a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.766+0000 7fbfec92a700 1 --2- 172.21.15.129:0/1048699281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe8102370 0x7fbfe8102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.766+0000 7fbfec92a700 1 --2- 172.21.15.129:0/1048699281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8103570 0x7fbfe81039d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.766+0000 7fbfec92a700 1 -- 172.21.15.129:0/1048699281 >> 172.21.15.129:0/1048699281 conn(0x7fbfe80fd910 msgr2=0x7fbfe80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.766+0000 7fbfec92a700 1 -- 172.21.15.129:0/1048699281 shutdown_connections 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.766+0000 7fbfec92a700 1 -- 172.21.15.129:0/1048699281 wait complete. 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfec92a700 1 Processor -- start 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfec92a700 1 -- start start 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfec92a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8102370 0x7fbfe81062b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfec92a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbfe8103570 0x7fbfe8104950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfec92a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe8103f10 0x7fbfe8104e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfec92a700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe810df00 con 0x7fbfe8102370 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfec92a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe810dd80 con 0x7fbfe8103f10 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfec92a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe810e080 con 0x7fbfe8103570 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe659c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8102370 0x7fbfe81062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe659c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8102370 0x7fbfe81062b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40838/0 (socket says 172.21.15.129:40838) 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe659c700 1 -- 172.21.15.129:0/2911465647 learned_addr learned my addr 172.21.15.129:0/2911465647 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe6d9d700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe8103f10 0x7fbfe8104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe5d9b700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbfe8103570 0x7fbfe8104950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe659c700 1 -- 172.21.15.129:0/2911465647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbfe8103570 msgr2=0x7fbfe8104950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe659c700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbfe8103570 0x7fbfe8104950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe659c700 1 -- 172.21.15.129:0/2911465647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe8103f10 msgr2=0x7fbfe8104e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe659c700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe8103f10 0x7fbfe8104e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.767+0000 7fbfe659c700 1 -- 172.21.15.129:0/2911465647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe8105760 con 0x7fbfe8102370 2024-04-26T21:17:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.768+0000 7fbfe659c700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8102370 0x7fbfe81062b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00b500 tx=0x7fbfdc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.768+0000 7fbfd77fe700 1 -- 172.21.15.129:0/2911465647 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc015020 con 0x7fbfe8102370 2024-04-26T21:17:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.768+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe81a3810 con 0x7fbfe8102370 2024-04-26T21:17:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.768+0000 7fbfd77fe700 1 -- 172.21.15.129:0/2911465647 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfdc011ce0 con 0x7fbfe8102370 2024-04-26T21:17:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.768+0000 7fbfd77fe700 1 -- 172.21.15.129:0/2911465647 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc010c80 con 0x7fbfe8102370 2024-04-26T21:17:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.768+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe81a3da0 con 0x7fbfe8102370 2024-04-26T21:17:39.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.769+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe806a110 con 0x7fbfe8102370 2024-04-26T21:17:39.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.770+0000 7fbfd77fe700 1 -- 172.21.15.129:0/2911465647 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbfdc018020 con 0x7fbfe8102370 2024-04-26T21:17:39.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.770+0000 7fbfd77fe700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbfd00a3cd0 0x7fbfd00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.770+0000 7fbfe5d9b700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbfd00a3cd0 0x7fbfd00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.770+0000 7fbfd77fe700 1 -- 172.21.15.129:0/2911465647 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fbfdc0cfc20 con 0x7fbfe8102370 2024-04-26T21:17:39.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.771+0000 7fbfe5d9b700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbfd00a3cd0 0x7fbfd00a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009990 tx=0x7fbfd800b910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.775+0000 7fbfd77fe700 1 -- 172.21.15.129:0/2911465647 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbfdc01f020 con 0x7fbfe8102370 2024-04-26T21:17:39.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.979+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbfe806c080 con 0x7fbfd00a3cd0 2024-04-26T21:17:39.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.993+0000 7fbfd77fe700 1 -- 172.21.15.129:0/2911465647 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fbfe806c080 con 0x7fbfd00a3cd0 2024-04-26T21:17:39.994 INFO:teuthology.orchestra.run.smithi129.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T21:17:39.994 INFO:teuthology.orchestra.run.smithi129.stdout:mds.a smithi129 running (104s) 20s ago 103s 13.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 244246ed44da 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.b smithi150 running (101s) 34s ago 100s 25.8M - 19.0.0-3306-gbc612ffb 55dc54019ad8 d3dd5bd2c809 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.c smithi171 running (116s) 34s ago 115s 15.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 c058ac99f18f 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.d smithi129 running (2m) 20s ago 2m 14.0M - 19.0.0-3306-gbc612ffb 55dc54019ad8 5891d7cd780f 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.e smithi150 running (2m) 34s ago 2m 16.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 066ee3c811ea 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.f smithi171 running (119s) 34s ago 118s 13.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 93fd21c112dd 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.g smithi129 running (107s) 20s ago 106s 15.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 74fb377ae3ac 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.h smithi150 running (113s) 34s ago 112s 15.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 0c77778bd396 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.i smithi171 running (110s) 34s ago 109s 16.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 2f80c7340542 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.j smithi129 running (2m) 20s ago 2m 13.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 5187b1d16217 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.k smithi150 running (2m) 34s ago 2m 16.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 7d6369958e72 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mds.l smithi171 running (2m) 34s ago 2m 16.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 ed733b64450c 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mgr.x smithi129 *:9283,8765 running (9m) 20s ago 9m 516M - 19.0.0-3306-gbc612ffb 55dc54019ad8 74221cb1dd2d 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mgr.y smithi150 *:8443,8765 running (7m) 34s ago 7m 452M - 19.0.0-3306-gbc612ffb 55dc54019ad8 d9611832263d 2024-04-26T21:17:39.995 INFO:teuthology.orchestra.run.smithi129.stdout:mgr.z smithi171 *:8443,8765 running (7m) 34s ago 7m 454M - 19.0.0-3306-gbc612ffb 55dc54019ad8 22238896e915 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:mon.a smithi129 running (9m) 20s ago 9m 52.2M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 7a96e43a7575 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:mon.b smithi150 running (7m) 34s ago 7m 38.1M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 d42a09ce00db 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:mon.c smithi171 running (7m) 34s ago 7m 42.4M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 e0a14f2dafd7 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0 smithi129 running (7m) 20s ago 7m 57.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 583b5fa6eb59 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1 smithi150 running (6m) 34s ago 6m 59.0M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 e2ac813354f8 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2 smithi171 running (6m) 34s ago 6m 48.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 bedd3b483131 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3 smithi129 running (6m) 20s ago 6m 43.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 b2b4d4937e82 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4 smithi150 running (5m) 34s ago 5m 41.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 3043108fb981 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5 smithi171 running (5m) 34s ago 5m 43.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 f4e620d41876 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6 smithi129 running (4m) 20s ago 4m 41.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 4b5a9a9c78b6 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7 smithi150 running (4m) 34s ago 4m 52.8M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 93d420dc3a1d 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.8 smithi171 running (3m) 34s ago 3m 54.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 c3c530c177c7 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.9 smithi129 running (3m) 20s ago 3m 51.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 4f05870c9bef 2024-04-26T21:17:39.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.10 smithi150 running (3m) 34s ago 3m 43.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 590ee2f02365 2024-04-26T21:17:39.997 INFO:teuthology.orchestra.run.smithi129.stdout:osd.11 smithi171 running (2m) 34s ago 2m 40.1M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 e4c55b05fe60 2024-04-26T21:17:39.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbfd00a3cd0 msgr2=0x7fbfd00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbfd00a3cd0 0x7fbfd00a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009990 tx=0x7fbfd800b910 comp rx=0 tx=0).stop 2024-04-26T21:17:39.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8102370 msgr2=0x7fbfe81062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8102370 0x7fbfe81062b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00b500 tx=0x7fbfdc00b810 comp rx=0 tx=0).stop 2024-04-26T21:17:39.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 shutdown_connections 2024-04-26T21:17:39.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbfd00a3cd0 0x7fbfd00a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe8103f10 0x7fbfe8104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfe8102370 0x7fbfe81062b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 --2- 172.21.15.129:0/2911465647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbfe8103570 0x7fbfe8104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.995+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 >> 172.21.15.129:0/2911465647 conn(0x7fbfe80fd910 msgr2=0x7fbfe806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:39.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.996+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 shutdown_connections 2024-04-26T21:17:39.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:39.996+0000 7fbfec92a700 1 -- 172.21.15.129:0/2911465647 wait complete. 2024-04-26T21:17:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:40 smithi150 ceph-mon[32290]: pgmap v340: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:17:40.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:40 smithi129 ceph-mon[29237]: pgmap v340: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:17:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:40 smithi171 ceph-mon[33478]: pgmap v340: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:17:40.804 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T21:17:41.044 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:41.456 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:41 smithi129 ceph-mon[29237]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:41.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:41 smithi171 ceph-mon[33478]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:41.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:41 smithi150 ceph-mon[32290]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:42.686 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:42 smithi129 ceph-mon[29237]: pgmap v341: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:17:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:42 smithi171 ceph-mon[33478]: pgmap v341: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:17:42.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:42 smithi150 ceph-mon[32290]: pgmap v341: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:17:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.910+0000 7fb77c41c700 1 -- 172.21.15.129:0/3093249573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 msgr2=0x7fb7741027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.910+0000 7fb77c41c700 1 --2- 172.21.15.129:0/3093249573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 0x7fb7741027b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb76400b0d0 tx=0x7fb76400b3e0 comp rx=0 tx=0).stop 2024-04-26T21:17:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.910+0000 7fb77c41c700 1 -- 172.21.15.129:0/3093249573 shutdown_connections 2024-04-26T21:17:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.910+0000 7fb77c41c700 1 --2- 172.21.15.129:0/3093249573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb774103f30 0x7fb77410a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.910+0000 7fb77c41c700 1 --2- 172.21.15.129:0/3093249573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 0x7fb7741027b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.910+0000 7fb77c41c700 1 --2- 172.21.15.129:0/3093249573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb774103590 0x7fb7741039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.910+0000 7fb77c41c700 1 -- 172.21.15.129:0/3093249573 >> 172.21.15.129:0/3093249573 conn(0x7fb7740fd950 msgr2=0x7fb7740ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.910+0000 7fb77c41c700 1 -- 172.21.15.129:0/3093249573 shutdown_connections 2024-04-26T21:17:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.910+0000 7fb77c41c700 1 -- 172.21.15.129:0/3093249573 wait complete. 2024-04-26T21:17:42.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77c41c700 1 Processor -- start 2024-04-26T21:17:42.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77c41c700 1 -- start start 2024-04-26T21:17:42.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77c41c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 0x7fb7741985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:42.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77c41c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb774103590 0x7fb774198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:42.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77c41c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb774103f30 0x7fb77419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:42.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77c41c700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb77410de00 con 0x7fb774102390 2024-04-26T21:17:42.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77c41c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb77410dc80 con 0x7fb774103f30 2024-04-26T21:17:42.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77c41c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb77410df80 con 0x7fb774103590 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77a1b8700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 0x7fb7741985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77a1b8700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 0x7fb7741985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58176/0 (socket says 172.21.15.129:58176) 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77a1b8700 1 -- 172.21.15.129:0/1460848806 learned_addr learned my addr 172.21.15.129:0/1460848806 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb77a9b9700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb774103f30 0x7fb77419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.911+0000 7fb7799b7700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb774103590 0x7fb774198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb77a1b8700 1 -- 172.21.15.129:0/1460848806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb774103590 msgr2=0x7fb774198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb77a1b8700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb774103590 0x7fb774198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb77a1b8700 1 -- 172.21.15.129:0/1460848806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb774103f30 msgr2=0x7fb77419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb77a1b8700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb774103f30 0x7fb77419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb77a1b8700 1 -- 172.21.15.129:0/1460848806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77406bb70 con 0x7fb774102390 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb7799b7700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb774103590 0x7fb774198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb77a1b8700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 0x7fb7741985c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb76400b770 tx=0x7fb764004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb7737fe700 1 -- 172.21.15.129:0/1460848806 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76401f070 con 0x7fb774102390 2024-04-26T21:17:42.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77406be00 con 0x7fb774102390 2024-04-26T21:17:42.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb7737fe700 1 -- 172.21.15.129:0/1460848806 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb764007cc0 con 0x7fb774102390 2024-04-26T21:17:42.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb7737fe700 1 -- 172.21.15.129:0/1460848806 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76401b830 con 0x7fb774102390 2024-04-26T21:17:42.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.912+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77406c330 con 0x7fb774102390 2024-04-26T21:17:42.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.913+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77406a110 con 0x7fb774102390 2024-04-26T21:17:42.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.914+0000 7fb7737fe700 1 -- 172.21.15.129:0/1460848806 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb764022070 con 0x7fb774102390 2024-04-26T21:17:42.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.914+0000 7fb7737fe700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb75c0a3cd0 0x7fb75c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:42.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.915+0000 7fb7799b7700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb75c0a3cd0 0x7fb75c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:42.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.915+0000 7fb7737fe700 1 -- 172.21.15.129:0/1460848806 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb76400f020 con 0x7fb774102390 2024-04-26T21:17:42.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.915+0000 7fb7799b7700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb75c0a3cd0 0x7fb75c0a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb768005fd0 tx=0x7fb768005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:42.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:42.919+0000 7fb7737fe700 1 -- 172.21.15.129:0/1460848806 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb764029020 con 0x7fb774102390 2024-04-26T21:17:43.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.125+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb77406c500 con 0x7fb75c0a3cd0 2024-04-26T21:17:43.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.131+0000 7fb7737fe700 1 -- 172.21.15.129:0/1460848806 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb77406c500 con 0x7fb75c0a3cd0 2024-04-26T21:17:43.132 INFO:teuthology.orchestra.run.smithi129.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T21:17:43.132 INFO:teuthology.orchestra.run.smithi129.stdout:mds.all 12/12 37s ago 2m smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-26T21:17:43.132 INFO:teuthology.orchestra.run.smithi129.stdout:mgr 3/3 37s ago 7m smithi129=x;smithi150=y;smithi171=z;count:3 2024-04-26T21:17:43.132 INFO:teuthology.orchestra.run.smithi129.stdout:mon 3/3 37s ago 8m smithi129:172.21.15.129=a;smithi150:172.21.15.150=b;smithi171:172.21.15.171=c;count:3 2024-04-26T21:17:43.133 INFO:teuthology.orchestra.run.smithi129.stdout:osd 12 37s ago - 2024-04-26T21:17:43.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb75c0a3cd0 msgr2=0x7fb75c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb75c0a3cd0 0x7fb75c0a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb768005fd0 tx=0x7fb768005dc0 comp rx=0 tx=0).stop 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 msgr2=0x7fb7741985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 0x7fb7741985c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb76400b770 tx=0x7fb764004d90 comp rx=0 tx=0).stop 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 shutdown_connections 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb75c0a3cd0 0x7fb75c0a6190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb774103f30 0x7fb77419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb774102390 0x7fb7741985c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 --2- 172.21.15.129:0/1460848806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb774103590 0x7fb774198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 >> 172.21.15.129:0/1460848806 conn(0x7fb7740fd950 msgr2=0x7fb774106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 shutdown_connections 2024-04-26T21:17:43.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:43.133+0000 7fb77c41c700 1 -- 172.21.15.129:0/1460848806 wait complete. 2024-04-26T21:17:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:43 smithi129 ceph-mon[29237]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:43 smithi129 ceph-mon[29237]: pgmap v342: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:17:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:43 smithi171 ceph-mon[33478]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:43 smithi171 ceph-mon[33478]: pgmap v342: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:17:43.835 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T21:17:43.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:43 smithi150 ceph-mon[32290]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:43.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:43 smithi150 ceph-mon[32290]: pgmap v342: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:17:44.072 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:45.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.938+0000 7f515937d700 1 -- 172.21.15.129:0/1689742911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154103f00 msgr2=0x7f51541062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:45.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.938+0000 7f515937d700 1 --2- 172.21.15.129:0/1689742911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154103f00 0x7f51541062f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5144009990 tx=0x7f5144009ca0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.938+0000 7f515937d700 1 -- 172.21.15.129:0/1689742911 shutdown_connections 2024-04-26T21:17:45.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.938+0000 7f515937d700 1 --2- 172.21.15.129:0/1689742911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5154109160 0x7f515410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.938+0000 7f515937d700 1 --2- 172.21.15.129:0/1689742911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154103f00 0x7f51541062f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.938+0000 7f515937d700 1 --2- 172.21.15.129:0/1689742911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5154106830 0x7f5154108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.938+0000 7f515937d700 1 -- 172.21.15.129:0/1689742911 >> 172.21.15.129:0/1689742911 conn(0x7f51540fd910 msgr2=0x7f51540ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.938+0000 7f515937d700 1 -- 172.21.15.129:0/1689742911 shutdown_connections 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.938+0000 7f515937d700 1 -- 172.21.15.129:0/1689742911 wait complete. 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f515937d700 1 Processor -- start 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f515937d700 1 -- start start 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f515937d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5154103f00 0x7f5154199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f515937d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5154106830 0x7f5154199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f515937d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154109160 0x7f51541a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f515937d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f515410eff0 con 0x7f5154109160 2024-04-26T21:17:45.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f515937d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f515410ee70 con 0x7f5154103f00 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f515937d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f515410f170 con 0x7f5154106830 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f51537fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154109160 0x7f51541a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f51537fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154109160 0x7f51541a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58200/0 (socket says 172.21.15.129:58200) 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f51537fe700 1 -- 172.21.15.129:0/770621347 learned_addr learned my addr 172.21.15.129:0/770621347 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f5152ffd700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5154103f00 0x7f5154199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f514a7fc700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5154106830 0x7f5154199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f51537fe700 1 -- 172.21.15.129:0/770621347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5154106830 msgr2=0x7f5154199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f51537fe700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5154106830 0x7f5154199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f51537fe700 1 -- 172.21.15.129:0/770621347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5154103f00 msgr2=0x7f5154199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f51537fe700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5154103f00 0x7f5154199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f51537fe700 1 -- 172.21.15.129:0/770621347 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51541a13a0 con 0x7f5154109160 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.939+0000 7f5152ffd700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5154103f00 0x7f5154199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.940+0000 7f51537fe700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154109160 0x7f51541a0d00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f514c00ba10 tx=0x7f514c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:45.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.940+0000 7f5150ff9700 1 -- 172.21.15.129:0/770621347 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c019070 con 0x7f5154109160 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.940+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51541a1690 con 0x7f5154109160 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.940+0000 7f5150ff9700 1 -- 172.21.15.129:0/770621347 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f514c014440 con 0x7f5154109160 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.940+0000 7f5150ff9700 1 -- 172.21.15.129:0/770621347 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c00ce60 con 0x7f5154109160 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.940+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51541a1ca0 con 0x7f5154109160 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.941+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515406a110 con 0x7f5154109160 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.942+0000 7f5150ff9700 1 -- 172.21.15.129:0/770621347 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f514c01c070 con 0x7f5154109160 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.942+0000 7f5150ff9700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f51400a3cd0 0x7f51400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.943+0000 7f5150ff9700 1 -- 172.21.15.129:0/770621347 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f514c0cf810 con 0x7f5154109160 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.943+0000 7f5152ffd700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f51400a3cd0 0x7f51400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:45.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.943+0000 7f5152ffd700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f51400a3cd0 0x7f51400a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f514400b5c0 tx=0x7f5144005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:45.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:45.947+0000 7f5150ff9700 1 -- 172.21.15.129:0/770621347 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f514c09c220 con 0x7f5154109160 2024-04-26T21:17:46.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.154+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5154063860 con 0x7f51400a3cd0 2024-04-26T21:17:46.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.155+0000 7f5150ff9700 1 -- 172.21.15.129:0/770621347 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5154063860 con 0x7f51400a3cd0 2024-04-26T21:17:46.156 INFO:teuthology.orchestra.run.smithi129.stdout:HOST ADDR LABELS STATUS 2024-04-26T21:17:46.157 INFO:teuthology.orchestra.run.smithi129.stdout:smithi129 172.21.15.129 2024-04-26T21:17:46.157 INFO:teuthology.orchestra.run.smithi129.stdout:smithi150 172.21.15.150 2024-04-26T21:17:46.157 INFO:teuthology.orchestra.run.smithi129.stdout:smithi171 172.21.15.171 2024-04-26T21:17:46.157 INFO:teuthology.orchestra.run.smithi129.stdout:3 hosts in cluster 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.157+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f51400a3cd0 msgr2=0x7f51400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.157+0000 7f515937d700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f51400a3cd0 0x7f51400a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f514400b5c0 tx=0x7f5144005fd0 comp rx=0 tx=0).stop 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.157+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154109160 msgr2=0x7f51541a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.157+0000 7f515937d700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154109160 0x7f51541a0d00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f514c00ba10 tx=0x7f514c00bdd0 comp rx=0 tx=0).stop 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.158+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 shutdown_connections 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.158+0000 7f515937d700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f51400a3cd0 0x7f51400a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.158+0000 7f515937d700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5154109160 0x7f51541a0d00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.158+0000 7f515937d700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5154103f00 0x7f5154199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.158+0000 7f515937d700 1 --2- 172.21.15.129:0/770621347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5154106830 0x7f5154199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.158+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 >> 172.21.15.129:0/770621347 conn(0x7f51540fd910 msgr2=0x7f51541049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.158+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 shutdown_connections 2024-04-26T21:17:46.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:46.158+0000 7f515937d700 1 -- 172.21.15.129:0/770621347 wait complete. 2024-04-26T21:17:46.469 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:46 smithi129 ceph-mon[29237]: pgmap v343: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:46 smithi150 ceph-mon[32290]: pgmap v343: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:46.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:46 smithi171 ceph-mon[33478]: pgmap v343: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:46.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T21:17:47.227 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:47.620 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:47 smithi129 ceph-mon[29237]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:47.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:47 smithi150 ceph-mon[32290]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:47 smithi171 ceph-mon[33478]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:48 smithi150 ceph-mon[32290]: pgmap v344: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:48.695 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:48 smithi129 ceph-mon[29237]: pgmap v344: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:48.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:48 smithi171 ceph-mon[33478]: pgmap v344: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:49.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.073+0000 7fb267e8f700 1 -- 172.21.15.129:0/3419590401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103f10 msgr2=0x7fb26010a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:49.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.073+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3419590401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103f10 0x7fb26010a7c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb25800b0d0 tx=0x7fb25800b3e0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.073+0000 7fb267e8f700 1 -- 172.21.15.129:0/3419590401 shutdown_connections 2024-04-26T21:17:49.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.073+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3419590401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103f10 0x7fb26010a7c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.073+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3419590401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb260102370 0x7fb260102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.073+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3419590401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260103570 0x7fb2601039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.073+0000 7fb267e8f700 1 -- 172.21.15.129:0/3419590401 >> 172.21.15.129:0/3419590401 conn(0x7fb2600fd8f0 msgr2=0x7fb2600ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.073+0000 7fb267e8f700 1 -- 172.21.15.129:0/3419590401 shutdown_connections 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.073+0000 7fb267e8f700 1 -- 172.21.15.129:0/3419590401 wait complete. 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb267e8f700 1 Processor -- start 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb267e8f700 1 -- start start 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb267e8f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb260102370 0x7fb2601062b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb267e8f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103570 0x7fb260104950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb267e8f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260103f10 0x7fb260104e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb267e8f700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb26010de60 con 0x7fb260103570 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb267e8f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb26010dce0 con 0x7fb260102370 2024-04-26T21:17:49.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb267e8f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb26010dfe0 con 0x7fb260103f10 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb26542a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103570 0x7fb260104950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb26542a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103570 0x7fb260104950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58226/0 (socket says 172.21.15.129:58226) 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.074+0000 7fb26542a700 1 -- 172.21.15.129:0/3554910101 learned_addr learned my addr 172.21.15.129:0/3554910101 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb26642c700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260103f10 0x7fb260104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb265c2b700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb260102370 0x7fb2601062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb26542a700 1 -- 172.21.15.129:0/3554910101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260103f10 msgr2=0x7fb260104e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb26542a700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260103f10 0x7fb260104e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb26542a700 1 -- 172.21.15.129:0/3554910101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb260102370 msgr2=0x7fb2601062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb26542a700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb260102370 0x7fb2601062b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb26542a700 1 -- 172.21.15.129:0/3554910101 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb260105760 con 0x7fb260103570 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb26642c700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260103f10 0x7fb260104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb26542a700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103570 0x7fb260104950 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb25400b530 tx=0x7fb25400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:49.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb252ffd700 1 -- 172.21.15.129:0/3554910101 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254015020 con 0x7fb260103570 2024-04-26T21:17:49.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb252ffd700 1 -- 172.21.15.129:0/3554910101 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb254011ce0 con 0x7fb260103570 2024-04-26T21:17:49.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb252ffd700 1 -- 172.21.15.129:0/3554910101 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254010d20 con 0x7fb260103570 2024-04-26T21:17:49.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2601a3660 con 0x7fb260103570 2024-04-26T21:17:49.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.075+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2601a3c40 con 0x7fb260103570 2024-04-26T21:17:49.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.076+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26006a110 con 0x7fb260103570 2024-04-26T21:17:49.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.077+0000 7fb252ffd700 1 -- 172.21.15.129:0/3554910101 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb254018020 con 0x7fb260103570 2024-04-26T21:17:49.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.078+0000 7fb252ffd700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb24c0a3d50 0x7fb24c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:49.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.078+0000 7fb265c2b700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb24c0a3d50 0x7fb24c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:49.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.078+0000 7fb252ffd700 1 -- 172.21.15.129:0/3554910101 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb2540cfcc0 con 0x7fb260103570 2024-04-26T21:17:49.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.078+0000 7fb265c2b700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb24c0a3d50 0x7fb24c0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005fd0 tx=0x7fb25c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:49.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.082+0000 7fb252ffd700 1 -- 172.21.15.129:0/3554910101 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb25401f020 con 0x7fb260103570 2024-04-26T21:17:49.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.290+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb2600ff9f0 con 0x7fb24c0a3d50 2024-04-26T21:17:49.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.292+0000 7fb252ffd700 1 -- 172.21.15.129:0/3554910101 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb2600ff9f0 con 0x7fb24c0a3d50 2024-04-26T21:17:49.293 INFO:teuthology.orchestra.run.smithi129.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T21:17:49.293 INFO:teuthology.orchestra.run.smithi129.stdout:smithi129 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300HY400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T21:17:49.293 INFO:teuthology.orchestra.run.smithi129.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T21:17:49.293 INFO:teuthology.orchestra.run.smithi129.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-26T21:17:49.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb24c0a3d50 msgr2=0x7fb24c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb24c0a3d50 0x7fb24c0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005fd0 tx=0x7fb25c005e50 comp rx=0 tx=0).stop 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103570 msgr2=0x7fb260104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103570 0x7fb260104950 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb25400b530 tx=0x7fb25400b8f0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 shutdown_connections 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb24c0a3d50 0x7fb24c0a6210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260103f10 0x7fb260104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb260102370 0x7fb2601062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 --2- 172.21.15.129:0/3554910101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb260103570 0x7fb260104950 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 >> 172.21.15.129:0/3554910101 conn(0x7fb2600fd8f0 msgr2=0x7fb2600ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 shutdown_connections 2024-04-26T21:17:49.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:49.294+0000 7fb267e8f700 1 -- 172.21.15.129:0/3554910101 wait complete. 2024-04-26T21:17:50.128 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:17:50.140 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi129.front.sepia.ceph.com 2024-04-26T21:17:50.140 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T21:17:50.379 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:50 smithi150 ceph-mon[32290]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:50 smithi150 ceph-mon[32290]: pgmap v345: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:50.634 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:50 smithi129 ceph-mon[29237]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:50.635 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:50 smithi129 ceph-mon[29237]: pgmap v345: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:50 smithi171 ceph-mon[33478]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:17:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:50 smithi171 ceph-mon[33478]: pgmap v345: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:51.960 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:51 smithi129 ceph-mon[29237]: pgmap v346: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:51.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:51 smithi171 ceph-mon[33478]: pgmap v346: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:52.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:51 smithi150 ceph-mon[32290]: pgmap v346: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:52.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.214+0000 7f13a002f700 1 -- 172.21.15.129:0/1776342014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 msgr2=0x7f13981039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:52.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.214+0000 7f13a002f700 1 --2- 172.21.15.129:0/1776342014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 0x7f13981039e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1388009990 tx=0x7f1388009ca0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.214+0000 7f13a002f700 1 -- 172.21.15.129:0/1776342014 shutdown_connections 2024-04-26T21:17:52.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.214+0000 7f13a002f700 1 --2- 172.21.15.129:0/1776342014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1398103f20 0x7f139810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.214+0000 7f13a002f700 1 --2- 172.21.15.129:0/1776342014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1398102380 0x7f13981027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.214+0000 7f13a002f700 1 --2- 172.21.15.129:0/1776342014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 0x7f13981039e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.214+0000 7f13a002f700 1 -- 172.21.15.129:0/1776342014 >> 172.21.15.129:0/1776342014 conn(0x7f13980fd920 msgr2=0x7f13980ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.214+0000 7f13a002f700 1 -- 172.21.15.129:0/1776342014 shutdown_connections 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.214+0000 7f13a002f700 1 -- 172.21.15.129:0/1776342014 wait complete. 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f13a002f700 1 Processor -- start 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f13a002f700 1 -- start start 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f13a002f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1398102380 0x7f1398199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f13a002f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 0x7f1398199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f13a002f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1398103f20 0x7f13981a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f13a002f700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f139810ef90 con 0x7f1398103580 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f13a002f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f139810ee10 con 0x7f1398102380 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f13a002f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f139810f110 con 0x7f1398103f20 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f139d5ca700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 0x7f1398199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f139d5ca700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 0x7f1398199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58242/0 (socket says 172.21.15.129:58242) 2024-04-26T21:17:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.215+0000 7f139d5ca700 1 -- 172.21.15.129:0/4079461114 learned_addr learned my addr 172.21.15.129:0/4079461114 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f139ddcb700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1398102380 0x7f1398199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f139d5ca700 1 -- 172.21.15.129:0/4079461114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1398103f20 msgr2=0x7f13981a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f139d5ca700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1398103f20 0x7f13981a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f139d5ca700 1 -- 172.21.15.129:0/4079461114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1398102380 msgr2=0x7f1398199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f139d5ca700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1398102380 0x7f1398199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f139d5ca700 1 -- 172.21.15.129:0/4079461114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13981a1290 con 0x7f1398103580 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f139ddcb700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1398102380 0x7f1398199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f139d5ca700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 0x7f1398199b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f13880059d0 tx=0x7f1388009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f138effd700 1 -- 172.21.15.129:0/4079461114 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388027070 con 0x7f1398103580 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13981a1520 con 0x7f1398103580 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f138effd700 1 -- 172.21.15.129:0/4079461114 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1388004080 con 0x7f1398103580 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f138effd700 1 -- 172.21.15.129:0/4079461114 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388022cf0 con 0x7f1398103580 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.216+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13981a1980 con 0x7f1398103580 2024-04-26T21:17:52.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.217+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139806a110 con 0x7f1398103580 2024-04-26T21:17:52.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.218+0000 7f138effd700 1 -- 172.21.15.129:0/4079461114 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f138802a070 con 0x7f1398103580 2024-04-26T21:17:52.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.219+0000 7f138effd700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f13840a3cd0 0x7f13840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:52.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.219+0000 7f139ddcb700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f13840a3cd0 0x7f13840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:52.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.219+0000 7f138effd700 1 -- 172.21.15.129:0/4079461114 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f13880d1c70 con 0x7f1398103580 2024-04-26T21:17:52.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.219+0000 7f139ddcb700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f13840a3cd0 0x7f13840a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1394005fd0 tx=0x7f1394005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:52.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.222+0000 7f138effd700 1 -- 172.21.15.129:0/4079461114 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1388031070 con 0x7f1398103580 2024-04-26T21:17:52.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.463+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f139802e560 con 0x7f1398103580 2024-04-26T21:17:52.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.463+0000 7f138effd700 1 -- 172.21.15.129:0/4079461114 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+3020 (secure 0 0 0) 0x7f1388013070 con 0x7f1398103580 2024-04-26T21:17:52.464 INFO:teuthology.orchestra.run.smithi129.stdout:e19 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:legacy client fscid: 1 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:Filesystem 'cephfs' (1) 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:fs_name cephfs 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 19 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-04-26T21:16:03.677644+0000 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:modified 2024-04-26T21:16:08.015827+0000 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:tableserver 0 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:root 0 2024-04-26T21:17:52.465 INFO:teuthology.orchestra.run.smithi129.stdout:session_timeout 300 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:session_autoclose 300 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:max_file_size 1099511627776 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:max_xattr_size 65536 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:required_client_features {} 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:last_failure 0 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:last_failure_osd_epoch 0 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:max_mds 1 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:in 0 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:up {0=24497} 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:failed 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:damaged 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:stopped 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:data_pools [3] 2024-04-26T21:17:52.466 INFO:teuthology.orchestra.run.smithi129.stdout:metadata_pool 2 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:inline_data disabled 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:balancer 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:bal_rank_mask -1 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:standby_count_wanted 0 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:qdb_cluster leader: 24497 members: 24497 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.b{0:24497} state up:active seq 3 addr [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.i{0:24484} state up:standby-replay seq 1 addr [v2:172.21.15.171:6838/2157967831,v1:172.21.15.171:6839/2157967831] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:Standby daemons: 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.j{-1:14583} state up:standby seq 1 addr [v2:172.21.15.129:6834/4106276863,v1:172.21.15.129:6835/4106276863] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.d{-1:14595} state up:standby seq 1 addr [v2:172.21.15.129:6836/277014387,v1:172.21.15.129:6837/277014387] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.g{-1:14661} state up:standby seq 1 addr [v2:172.21.15.129:6838/556190328,v1:172.21.15.129:6839/556190328] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.467 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.a{-1:14679} state up:standby seq 1 addr [v2:172.21.15.129:6840/861528806,v1:172.21.15.129:6841/861528806] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.468 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.l{-1:24406} state up:standby seq 1 addr [v2:172.21.15.171:6832/1986119815,v1:172.21.15.171:6833/1986119815] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.468 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.k{-1:24428} state up:standby seq 1 addr [v2:172.21.15.150:6832/1663214837,v1:172.21.15.150:6833/1663214837] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.468 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.f{-1:24451} state up:standby seq 1 addr [v2:172.21.15.171:6834/3940507923,v1:172.21.15.171:6835/3940507923] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.468 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.e{-1:24452} state up:standby seq 1 addr [v2:172.21.15.150:6834/2260848843,v1:172.21.15.150:6835/2260848843] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.468 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.c{-1:24463} state up:standby seq 1 addr [v2:172.21.15.171:6836/2148122468,v1:172.21.15.171:6837/2148122468] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.468 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.h{-1:24470} state up:standby seq 1 addr [v2:172.21.15.150:6836/1663379365,v1:172.21.15.150:6837/1663379365] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.465+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f13840a3cd0 msgr2=0x7f13840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.465+0000 7f13a002f700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f13840a3cd0 0x7f13840a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1394005fd0 tx=0x7f1394005f00 comp rx=0 tx=0).stop 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.465+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 msgr2=0x7f1398199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.465+0000 7f13a002f700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 0x7f1398199b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f13880059d0 tx=0x7f1388009f90 comp rx=0 tx=0).stop 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.466+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 shutdown_connections 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.466+0000 7f13a002f700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f13840a3cd0 0x7f13840a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.466+0000 7f13a002f700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1398103f20 0x7f13981a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.466+0000 7f13a002f700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1398102380 0x7f1398199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.466+0000 7f13a002f700 1 --2- 172.21.15.129:0/4079461114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1398103580 0x7f1398199b70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.466+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 >> 172.21.15.129:0/4079461114 conn(0x7f13980fd920 msgr2=0x7f13981069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.466+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 shutdown_connections 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:52.466+0000 7f13a002f700 1 -- 172.21.15.129:0/4079461114 wait complete. 2024-04-26T21:17:52.469 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:17:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:52 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/4079461114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:17:52.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:52 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/4079461114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:17:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:52 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/4079461114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:17:53.157 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T21:17:53.397 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:17:53.772 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:53 smithi129 ceph-mon[29237]: pgmap v347: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:53.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:53 smithi171 ceph-mon[33478]: pgmap v347: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:54.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:53 smithi150 ceph-mon[32290]: pgmap v347: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:55.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.197+0000 7f0174095700 1 -- 172.21.15.129:0/1708840917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c1025e0 msgr2=0x7f016c102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.197+0000 7f0174095700 1 --2- 172.21.15.129:0/1708840917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c1025e0 0x7f016c102a60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f015c009990 tx=0x7f015c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.197+0000 7f0174095700 1 -- 172.21.15.129:0/1708840917 shutdown_connections 2024-04-26T21:17:55.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.197+0000 7f0174095700 1 --2- 172.21.15.129:0/1708840917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c102fa0 0x7f016c10a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.197+0000 7f0174095700 1 --2- 172.21.15.129:0/1708840917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f016c103a10 0x7f016c103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.197+0000 7f0174095700 1 --2- 172.21.15.129:0/1708840917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c1025e0 0x7f016c102a60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.197+0000 7f0174095700 1 -- 172.21.15.129:0/1708840917 >> 172.21.15.129:0/1708840917 conn(0x7f016c0fd090 msgr2=0x7f016c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 -- 172.21.15.129:0/1708840917 shutdown_connections 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 -- 172.21.15.129:0/1708840917 wait complete. 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 Processor -- start 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 -- start start 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c1025e0 0x7f016c1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f016c102fa0 0x7f016c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c103a10 0x7f016c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f016c10de00 con 0x7f016c103a10 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f016c10dc80 con 0x7f016c1025e0 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0174095700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f016c10df80 con 0x7f016c102fa0 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.198+0000 7f0172632700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c103a10 0x7f016c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0172632700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c103a10 0x7f016c1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33228/0 (socket says 172.21.15.129:33228) 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0172632700 1 -- 172.21.15.129:0/769191048 learned_addr learned my addr 172.21.15.129:0/769191048 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:55.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0171e31700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c1025e0 0x7f016c1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0171630700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f016c102fa0 0x7f016c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0172632700 1 -- 172.21.15.129:0/769191048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f016c102fa0 msgr2=0x7f016c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0172632700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f016c102fa0 0x7f016c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0172632700 1 -- 172.21.15.129:0/769191048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c1025e0 msgr2=0x7f016c1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0172632700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c1025e0 0x7f016c1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0172632700 1 -- 172.21.15.129:0/769191048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c1a1130 con 0x7f016c103a10 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0172632700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c103a10 0x7f016c1a0a90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0164007f10 tx=0x7f016400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0162ffd700 1 -- 172.21.15.129:0/769191048 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164004bb0 con 0x7f016c103a10 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0162ffd700 1 -- 172.21.15.129:0/769191048 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0164004d50 con 0x7f016c103a10 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c1a1420 con 0x7f016c103a10 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0162ffd700 1 -- 172.21.15.129:0/769191048 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640058e0 con 0x7f016c103a10 2024-04-26T21:17:55.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.199+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c1a1960 con 0x7f016c103a10 2024-04-26T21:17:55.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.200+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c06a110 con 0x7f016c103a10 2024-04-26T21:17:55.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.201+0000 7f0162ffd700 1 -- 172.21.15.129:0/769191048 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f016401a020 con 0x7f016c103a10 2024-04-26T21:17:55.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.202+0000 7f0162ffd700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01580a3cd0 0x7f01580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.202+0000 7f0162ffd700 1 -- 172.21.15.129:0/769191048 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f01640d0330 con 0x7f016c103a10 2024-04-26T21:17:55.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.202+0000 7f0171e31700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01580a3cd0 0x7f01580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.202+0000 7f0171e31700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01580a3cd0 0x7f01580a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f016800ac50 tx=0x7f016800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:55.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.206+0000 7f0162ffd700 1 -- 172.21.15.129:0/769191048 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0164010070 con 0x7f016c103a10 2024-04-26T21:17:55.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.409+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f016c1a1d30 con 0x7f016c103a10 2024-04-26T21:17:55.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.410+0000 7f0162ffd700 1 -- 172.21.15.129:0/769191048 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5337 (secure 0 0 0) 0x7f016409cba0 con 0x7f016c103a10 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 88 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:fsid eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-04-26T21:07:57.720565+0000 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:modified 2024-04-26T21:16:56.355853+0000 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:crush_version 26 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:full_ratio 0.95 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:backfillfull_ratio 0.9 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:nearfull_ratio 0.85 2024-04-26T21:17:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:require_min_compat_client luminous 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:min_compat_client luminous 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:require_osd_release squid 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:stretch_mode_enabled false 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:max_osd 12 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6802/3232878544,v1:172.21.15.129:6803/3232878544] [v2:172.21.15.129:6804/3232878544,v1:172.21.15.129:6805/3232878544] exists,up 03a78174-76a0-4fbb-baf7-222c97cf105c 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/735494308,v1:172.21.15.150:6801/735494308] [v2:172.21.15.150:6802/735494308,v1:172.21.15.150:6803/735494308] exists,up f8567f6e-0353-446c-8272-3c2d98c95515 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/1157402243,v1:172.21.15.171:6801/1157402243] [v2:172.21.15.171:6802/1157402243,v1:172.21.15.171:6803/1157402243] exists,up 984f85b5-efab-46bb-99c0-e94a53cb512b 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3 up in weight 1 up_from 26 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6810/347897303,v1:172.21.15.129:6811/347897303] [v2:172.21.15.129:6812/347897303,v1:172.21.15.129:6813/347897303] exists,up 04e52e54-d5ed-417d-957b-e951ff05b920 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/155182298,v1:172.21.15.150:6809/155182298] [v2:172.21.15.150:6810/155182298,v1:172.21.15.150:6811/155182298] exists,up fc2cdc94-fc55-4068-b8a0-0119fcb9dd39 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/1324309916,v1:172.21.15.171:6809/1324309916] [v2:172.21.15.171:6810/1324309916,v1:172.21.15.171:6811/1324309916] exists,up 09db0a71-d9ab-4959-9fb8-cee38f030d2d 2024-04-26T21:17:55.412 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6818/3546434113,v1:172.21.15.129:6819/3546434113] [v2:172.21.15.129:6820/3546434113,v1:172.21.15.129:6821/3546434113] exists,up 50c07cc9-ad4e-4283-9a5a-302eb0a948de 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/1143400463,v1:172.21.15.150:6817/1143400463] [v2:172.21.15.150:6818/1143400463,v1:172.21.15.150:6819/1143400463] exists,up 189c2c39-0395-43a9-b250-a1e88d350428 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/2306529186,v1:172.21.15.171:6817/2306529186] [v2:172.21.15.171:6818/2306529186,v1:172.21.15.171:6819/2306529186] exists,up 4382cd2f-fd00-45ef-9780-73271a167aa8 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6826/3561447119,v1:172.21.15.129:6827/3561447119] [v2:172.21.15.129:6828/3561447119,v1:172.21.15.129:6829/3561447119] exists,up c961ec13-aa37-4a66-b902-f36e159198b0 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/1187635399,v1:172.21.15.150:6825/1187635399] [v2:172.21.15.150:6826/1187635399,v1:172.21.15.150:6827/1187635399] exists,up 2eeeec10-5f31-4453-99bb-b705a557e632 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/2160214157,v1:172.21.15.171:6825/2160214157] [v2:172.21.15.171:6826/2160214157,v1:172.21.15.171:6827/2160214157] exists,up ae01372d-419f-4def-9fcd-3d49eb50b0e0 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 2.a [2,11] 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 2.1d [8,11] 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.c [3,9] 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.17 [4,6] 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.19 [3,8] 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.1b [5,9] 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.1d [4,7] 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.21 [5,9,4,6] 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.27 [4,8] 2024-04-26T21:17:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:6801/948723446 expires 2024-04-27T21:08:55.174047+0000 2024-04-26T21:17:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/3198429930 expires 2024-04-27T21:08:55.174047+0000 2024-04-26T21:17:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:6800/948723446 expires 2024-04-27T21:08:55.174047+0000 2024-04-26T21:17:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/905571244 expires 2024-04-27T21:08:55.174047+0000 2024-04-26T21:17:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:6800/1433037991 expires 2024-04-27T21:08:28.655399+0000 2024-04-26T21:17:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/1554108727 expires 2024-04-27T21:08:28.655399+0000 2024-04-26T21:17:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:6801/1433037991 expires 2024-04-27T21:08:28.655399+0000 2024-04-26T21:17:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/566668733 expires 2024-04-27T21:08:28.655399+0000 2024-04-26T21:17:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/841395546 expires 2024-04-27T21:08:28.655399+0000 2024-04-26T21:17:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/1281940584 expires 2024-04-27T21:08:55.174047+0000 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01580a3cd0 msgr2=0x7f01580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01580a3cd0 0x7f01580a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f016800ac50 tx=0x7f016800a380 comp rx=0 tx=0).stop 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c103a10 msgr2=0x7f016c1a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c103a10 0x7f016c1a0a90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0164007f10 tx=0x7f016400d3b0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 shutdown_connections 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01580a3cd0 0x7f01580a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f016c103a10 0x7f016c1a0a90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c1025e0 0x7f016c1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 --2- 172.21.15.129:0/769191048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f016c102fa0 0x7f016c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 >> 172.21.15.129:0/769191048 conn(0x7f016c0fd090 msgr2=0x7f016c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:55.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 shutdown_connections 2024-04-26T21:17:55.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:55.412+0000 7f0174095700 1 -- 172.21.15.129:0/769191048 wait complete. 2024-04-26T21:17:56.097 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T21:17:56.109 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:17:56.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.210+0000 7fcad8ba4640 1 -- 172.21.15.129:0/133657002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102290 msgr2=0x7fcad4102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.210+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/133657002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102290 0x7fcad4102710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009900 tx=0x7fcabc02a410 comp rx=0 tx=0).stop 2024-04-26T21:17:56.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.210+0000 7fcad8ba4640 1 -- 172.21.15.129:0/133657002 shutdown_connections 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.210+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/133657002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcad4102c50 0x7fcad41094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.210+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/133657002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102290 0x7fcad4102710 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.210+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/133657002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad41010a0 0x7fcad41014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.210+0000 7fcad8ba4640 1 -- 172.21.15.129:0/133657002 >> 172.21.15.129:0/133657002 conn(0x7fcad40fc610 msgr2=0x7fcad40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.210+0000 7fcad8ba4640 1 -- 172.21.15.129:0/133657002 shutdown_connections 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.210+0000 7fcad8ba4640 1 -- 172.21.15.129:0/133657002 wait complete. 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad8ba4640 1 Processor -- start 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad8ba4640 1 -- start start 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad8ba4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcad41010a0 0x7fcad410c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad8ba4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4102290 0x7fcad410ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad8ba4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102c50 0x7fcad410b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad8ba4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcad410e5c0 con 0x7fcad4102c50 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad8ba4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcad410e440 con 0x7fcad4102290 2024-04-26T21:17:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad8ba4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcad410e740 con 0x7fcad41010a0 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102c50 0x7fcad410b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102c50 0x7fcad410b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33246/0 (socket says 172.21.15.129:33246) 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad2d9d640 1 -- 172.21.15.129:0/557023550 learned_addr learned my addr 172.21.15.129:0/557023550 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcacbfff640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4102290 0x7fcad410ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad2d9d640 1 -- 172.21.15.129:0/557023550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcad41010a0 msgr2=0x7fcad410c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad2d9d640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcad41010a0 0x7fcad410c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad2d9d640 1 -- 172.21.15.129:0/557023550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4102290 msgr2=0x7fcad410ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad2d9d640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4102290 0x7fcad410ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad2d9d640 1 -- 172.21.15.129:0/557023550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad410bc40 con 0x7fcad4102c50 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcad2d9d640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102c50 0x7fcad410b3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00b970 tx=0x7fcacc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.211+0000 7fcacb7fe640 1 -- 172.21.15.129:0/557023550 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc019070 con 0x7fcad4102c50 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.212+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41a2de0 con 0x7fcad4102c50 2024-04-26T21:17:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.212+0000 7fcacb7fe640 1 -- 172.21.15.129:0/557023550 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcacc014440 con 0x7fcad4102c50 2024-04-26T21:17:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.212+0000 7fcacb7fe640 1 -- 172.21.15.129:0/557023550 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0075d0 con 0x7fcad4102c50 2024-04-26T21:17:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.212+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41a32a0 con 0x7fcad4102c50 2024-04-26T21:17:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.213+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad4108490 con 0x7fcad4102c50 2024-04-26T21:17:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.214+0000 7fcacb7fe640 1 -- 172.21.15.129:0/557023550 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fcacc01c070 con 0x7fcad4102c50 2024-04-26T21:17:56.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.214+0000 7fcacb7fe640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcac00a3c40 0x7fcac00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.214+0000 7fcad259c640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcac00a3c40 0x7fcac00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.214+0000 7fcacb7fe640 1 -- 172.21.15.129:0/557023550 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fcacc0cfe40 con 0x7fcad4102c50 2024-04-26T21:17:56.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.215+0000 7fcad259c640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcac00a3c40 0x7fcac00a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcac4005fd0 tx=0x7fcac400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:56.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.218+0000 7fcacb7fe640 1 -- 172.21.15.129:0/557023550 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcacc09c170 con 0x7fcad4102c50 2024-04-26T21:17:56.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.397+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcad402dd30 con 0x7fcad4102c50 2024-04-26T21:17:56.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.398+0000 7fcacb7fe640 1 -- 172.21.15.129:0/557023550 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7fcacc0a0470 con 0x7fcad4102c50 2024-04-26T21:17:56.400 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:56.400 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:17:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcac00a3c40 msgr2=0x7fcac00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcac00a3c40 0x7fcac00a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcac4005fd0 tx=0x7fcac400b410 comp rx=0 tx=0).stop 2024-04-26T21:17:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102c50 msgr2=0x7fcad410b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102c50 0x7fcad410b3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00b970 tx=0x7fcacc00be30 comp rx=0 tx=0).stop 2024-04-26T21:17:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 shutdown_connections 2024-04-26T21:17:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcac00a3c40 0x7fcac00a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4102290 0x7fcad410ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4102c50 0x7fcad410b3b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 --2- 172.21.15.129:0/557023550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcad41010a0 0x7fcad410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 >> 172.21.15.129:0/557023550 conn(0x7fcad40fc610 msgr2=0x7fcad4105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 shutdown_connections 2024-04-26T21:17:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.401+0000 7fcad8ba4640 1 -- 172.21.15.129:0/557023550 wait complete. 2024-04-26T21:17:56.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:16:01.621500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:16:02.592849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:17:56.414 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:17:56.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.514+0000 7ff66bc93640 1 -- 172.21.15.129:0/2723636235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664101860 msgr2=0x7ff664101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.514+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2723636235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664101860 0x7ff664101c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009900 tx=0x7ff65c02a410 comp rx=0 tx=0).stop 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.514+0000 7ff66bc93640 1 -- 172.21.15.129:0/2723636235 shutdown_connections 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.514+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2723636235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff664103410 0x7ff664109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.514+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2723636235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff664102a50 0x7ff664102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.514+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2723636235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664101860 0x7ff664101c80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.514+0000 7ff66bc93640 1 -- 172.21.15.129:0/2723636235 >> 172.21.15.129:0/2723636235 conn(0x7ff6640fcdd0 msgr2=0x7ff6640ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.514+0000 7ff66bc93640 1 -- 172.21.15.129:0/2723636235 shutdown_connections 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.514+0000 7ff66bc93640 1 -- 172.21.15.129:0/2723636235 wait complete. 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66bc93640 1 Processor -- start 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66bc93640 1 -- start start 2024-04-26T21:17:56.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66bc93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff664101860 0x7ff66406bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66bc93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664102a50 0x7ff66419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66bc93640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff664103410 0x7ff6641a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66bc93640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410d360 con 0x7ff664102a50 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66bc93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410d1e0 con 0x7ff664101860 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66bc93640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410d4e0 con 0x7ff664103410 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66922e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664102a50 0x7ff66419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66922e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664102a50 0x7ff66419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33264/0 (socket says 172.21.15.129:33264) 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66922e640 1 -- 172.21.15.129:0/2863001606 learned_addr learned my addr 172.21.15.129:0/2863001606 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66922e640 1 -- 172.21.15.129:0/2863001606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff664103410 msgr2=0x7ff6641a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66a230640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff664103410 0x7ff6641a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff669a2f640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff664101860 0x7ff66406bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66922e640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff664103410 0x7ff6641a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66922e640 1 -- 172.21.15.129:0/2863001606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff664101860 msgr2=0x7ff66406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66922e640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff664101860 0x7ff66406bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66922e640 1 -- 172.21.15.129:0/2863001606 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6641a2a20 con 0x7ff664102a50 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff669a2f640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff664101860 0x7ff66406bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.515+0000 7ff66a230640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff664103410 0x7ff6641a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.516+0000 7ff66922e640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664102a50 0x7ff66419fce0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d950 tx=0x7ff65400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.516+0000 7ff65affd640 1 -- 172.21.15.129:0/2863001606 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654017030 con 0x7ff664102a50 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.516+0000 7ff65affd640 1 -- 172.21.15.129:0/2863001606 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff654004020 con 0x7ff664102a50 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.516+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6641a2cb0 con 0x7ff664102a50 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.516+0000 7ff65affd640 1 -- 172.21.15.129:0/2863001606 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6540047a0 con 0x7ff664102a50 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.516+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6641a31f0 con 0x7ff664102a50 2024-04-26T21:17:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.517+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff664108bf0 con 0x7ff664102a50 2024-04-26T21:17:56.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.518+0000 7ff65affd640 1 -- 172.21.15.129:0/2863001606 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff654009a30 con 0x7ff664102a50 2024-04-26T21:17:56.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.518+0000 7ff65affd640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff6500a3cf0 0x7ff6500a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.518+0000 7ff669a2f640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff6500a3cf0 0x7ff6500a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.519+0000 7ff65affd640 1 -- 172.21.15.129:0/2863001606 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff6540d06a0 con 0x7ff664102a50 2024-04-26T21:17:56.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.519+0000 7ff669a2f640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff6500a3cf0 0x7ff6500a61b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006010 tx=0x7ff65c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:56.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.522+0000 7ff65affd640 1 -- 172.21.15.129:0/2863001606 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff65409cf90 con 0x7ff664102a50 2024-04-26T21:17:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:56 smithi150 ceph-mon[32290]: pgmap v348: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:56.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:56 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/769191048' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:17:56.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.698+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff66402dd20 con 0x7ff664102a50 2024-04-26T21:17:56.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.698+0000 7ff65affd640 1 -- 172.21.15.129:0/2863001606 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7ff65409c890 con 0x7ff664102a50 2024-04-26T21:17:56.700 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:17:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff6500a3cf0 msgr2=0x7ff6500a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff6500a3cf0 0x7ff6500a61b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006010 tx=0x7ff65c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664102a50 msgr2=0x7ff66419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664102a50 0x7ff66419fce0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d950 tx=0x7ff65400de10 comp rx=0 tx=0).stop 2024-04-26T21:17:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 shutdown_connections 2024-04-26T21:17:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff6500a3cf0 0x7ff6500a61b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff664103410 0x7ff6641a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff664102a50 0x7ff66419fce0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 --2- 172.21.15.129:0/2863001606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff664101860 0x7ff66406bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 >> 172.21.15.129:0/2863001606 conn(0x7ff6640fcdd0 msgr2=0x7ff664105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 shutdown_connections 2024-04-26T21:17:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.701+0000 7ff66bc93640 1 -- 172.21.15.129:0/2863001606 wait complete. 2024-04-26T21:17:56.712 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:56 smithi129 ceph-mon[29237]: pgmap v348: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:56.712 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:56 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/769191048' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:17:56.713 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:17:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:56 smithi171 ceph-mon[33478]: pgmap v348: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:56 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/769191048' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:17:56.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 -- 172.21.15.129:0/904755199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548101870 msgr2=0x7fd548101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 --2- 172.21.15.129:0/904755199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548101870 0x7fd548101c90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd538009900 tx=0x7fd53802a410 comp rx=0 tx=0).stop 2024-04-26T21:17:56.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 -- 172.21.15.129:0/904755199 shutdown_connections 2024-04-26T21:17:56.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 --2- 172.21.15.129:0/904755199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd548103420 0x7fd548109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 --2- 172.21.15.129:0/904755199 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd548102a60 0x7fd548102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 --2- 172.21.15.129:0/904755199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548101870 0x7fd548101c90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 -- 172.21.15.129:0/904755199 >> 172.21.15.129:0/904755199 conn(0x7fd5480fcde0 msgr2=0x7fd5480ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:56.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 -- 172.21.15.129:0/904755199 shutdown_connections 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 -- 172.21.15.129:0/904755199 wait complete. 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.813+0000 7fd54da6d640 1 Processor -- start 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd54da6d640 1 -- start start 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd54da6d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd548101870 0x7fd54806bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd54da6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548102a60 0x7fd54819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd54da6d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd548103420 0x7fd54819ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd54da6d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd54810d3f0 con 0x7fd548102a60 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd54da6d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd54810d270 con 0x7fd548101870 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd54da6d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd54810d570 con 0x7fd548103420 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd53ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548102a60 0x7fd54819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd53ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548102a60 0x7fd54819d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33282/0 (socket says 172.21.15.129:33282) 2024-04-26T21:17:56.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd53ffff640 1 -- 172.21.15.129:0/1772011629 learned_addr learned my addr 172.21.15.129:0/1772011629 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd5477fe640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd548101870 0x7fd54806bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd547fff640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd548103420 0x7fd54819ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd53ffff640 1 -- 172.21.15.129:0/1772011629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd548103420 msgr2=0x7fd54819ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd53ffff640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd548103420 0x7fd54819ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd53ffff640 1 -- 172.21.15.129:0/1772011629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd548101870 msgr2=0x7fd54806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd53ffff640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd548101870 0x7fd54806bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd53ffff640 1 -- 172.21.15.129:0/1772011629 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5481a08f0 con 0x7fd548102a60 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd547fff640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd548103420 0x7fd54819ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd53ffff640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548102a60 0x7fd54819d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd53000b4a0 tx=0x7fd53000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd5457fa640 1 -- 172.21.15.129:0/1772011629 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530014070 con 0x7fd548102a60 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd5457fa640 1 -- 172.21.15.129:0/1772011629 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd530010ac0 con 0x7fd548102a60 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5481a0b80 con 0x7fd548102a60 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd5457fa640 1 -- 172.21.15.129:0/1772011629 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530017c30 con 0x7fd548102a60 2024-04-26T21:17:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.814+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5481a10d0 con 0x7fd548102a60 2024-04-26T21:17:56.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.815+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd548108c50 con 0x7fd548102a60 2024-04-26T21:17:56.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.817+0000 7fd5457fa640 1 -- 172.21.15.129:0/1772011629 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd530017070 con 0x7fd548102a60 2024-04-26T21:17:56.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.817+0000 7fd5457fa640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5340a3cc0 0x7fd5340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:56.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.817+0000 7fd5477fe640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5340a3cc0 0x7fd5340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:56.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.817+0000 7fd5457fa640 1 -- 172.21.15.129:0/1772011629 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd5300cfb60 con 0x7fd548102a60 2024-04-26T21:17:56.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.818+0000 7fd5477fe640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5340a3cc0 0x7fd5340a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd538009230 tx=0x7fd5380314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:56.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.821+0000 7fd5457fa640 1 -- 172.21.15.129:0/1772011629 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd5300cfe20 con 0x7fd548102a60 2024-04-26T21:17:56.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.997+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd54802dad0 con 0x7fd548102a60 2024-04-26T21:17:56.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.997+0000 7fd5457fa640 1 -- 172.21.15.129:0/1772011629 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fd53009c4e0 con 0x7fd548102a60 2024-04-26T21:17:56.999 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-26T21:17:57.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5340a3cc0 msgr2=0x7fd5340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5340a3cc0 0x7fd5340a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd538009230 tx=0x7fd5380314a0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548102a60 msgr2=0x7fd54819d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548102a60 0x7fd54819d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd53000b4a0 tx=0x7fd53000b960 comp rx=0 tx=0).stop 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 shutdown_connections 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5340a3cc0 0x7fd5340a6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd548103420 0x7fd54819ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd548102a60 0x7fd54819d9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 --2- 172.21.15.129:0/1772011629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd548101870 0x7fd54806bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:56.999+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 >> 172.21.15.129:0/1772011629 conn(0x7fd5480fcde0 msgr2=0x7fd548105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.000+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 shutdown_connections 2024-04-26T21:17:57.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.000+0000 7fd54da6d640 1 -- 172.21.15.129:0/1772011629 wait complete. 2024-04-26T21:17:57.012 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:17:57.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.111+0000 7f07c196a640 1 -- 172.21.15.129:0/1619200076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc103420 msgr2=0x7f07bc109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.111+0000 7f07c196a640 1 --2- 172.21.15.129:0/1619200076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc103420 0x7f07bc109cb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009900 tx=0x7f07b402a410 comp rx=0 tx=0).stop 2024-04-26T21:17:57.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.111+0000 7f07c196a640 1 -- 172.21.15.129:0/1619200076 shutdown_connections 2024-04-26T21:17:57.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.111+0000 7f07c196a640 1 --2- 172.21.15.129:0/1619200076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc103420 0x7f07bc109cb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.111+0000 7f07c196a640 1 --2- 172.21.15.129:0/1619200076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07bc102a60 0x7f07bc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.111+0000 7f07c196a640 1 --2- 172.21.15.129:0/1619200076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07bc101870 0x7f07bc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.111+0000 7f07c196a640 1 -- 172.21.15.129:0/1619200076 >> 172.21.15.129:0/1619200076 conn(0x7f07bc0fcde0 msgr2=0x7f07bc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:57.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.111+0000 7f07c196a640 1 -- 172.21.15.129:0/1619200076 shutdown_connections 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.111+0000 7f07c196a640 1 -- 172.21.15.129:0/1619200076 wait complete. 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07c196a640 1 Processor -- start 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07c196a640 1 -- start start 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07c196a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07bc101870 0x7f07bc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07c196a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc102a60 0x7f07bc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07c196a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07bc103420 0x7f07bc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07c196a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc10d3a0 con 0x7f07bc102a60 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07c196a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc10d220 con 0x7f07bc103420 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07c196a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc10d520 con 0x7f07bc101870 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07ba7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc102a60 0x7f07bc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07ba7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc102a60 0x7f07bc197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33308/0 (socket says 172.21.15.129:33308) 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07ba7fc640 1 -- 172.21.15.129:0/1841223112 learned_addr learned my addr 172.21.15.129:0/1841223112 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:57.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.112+0000 7f07bb7fe640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07bc103420 0x7f07bc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07baffd640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07bc101870 0x7f07bc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07ba7fc640 1 -- 172.21.15.129:0/1841223112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07bc101870 msgr2=0x7f07bc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07ba7fc640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07bc101870 0x7f07bc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07ba7fc640 1 -- 172.21.15.129:0/1841223112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07bc103420 msgr2=0x7f07bc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07ba7fc640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07bc103420 0x7f07bc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07ba7fc640 1 -- 172.21.15.129:0/1841223112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc19f660 con 0x7f07bc102a60 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07ba7fc640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc102a60 0x7f07bc197f80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f07b000c970 tx=0x7f07b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07c0968640 1 -- 172.21.15.129:0/1841223112 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0007bb0 con 0x7f07bc102a60 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07c0968640 1 -- 172.21.15.129:0/1841223112 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b0007d50 con 0x7f07bc102a60 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc19f950 con 0x7f07bc102a60 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07c0968640 1 -- 172.21.15.129:0/1841223112 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b000f450 con 0x7f07bc102a60 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.113+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc19fe50 con 0x7f07bc102a60 2024-04-26T21:17:57.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.114+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07bc108bb0 con 0x7f07bc102a60 2024-04-26T21:17:57.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.115+0000 7f07c0968640 1 -- 172.21.15.129:0/1841223112 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f07b0004f20 con 0x7f07bc102a60 2024-04-26T21:17:57.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.116+0000 7f07c0968640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f07a40a3c40 0x7f07a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.116+0000 7f07baffd640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f07a40a3c40 0x7f07a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.116+0000 7f07c0968640 1 -- 172.21.15.129:0/1841223112 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f07b00d02f0 con 0x7f07bc102a60 2024-04-26T21:17:57.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.116+0000 7f07baffd640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f07a40a3c40 0x7f07a40a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005c40 tx=0x7f07ac005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:57.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.120+0000 7f07c0968640 1 -- 172.21.15.129:0/1841223112 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f07b0014030 con 0x7f07bc102a60 2024-04-26T21:17:57.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.297+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f07bc02dcf0 con 0x7f07bc102a60 2024-04-26T21:17:57.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.298+0000 7f07c0968640 1 -- 172.21.15.129:0/1841223112 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f07b009cb60 con 0x7f07bc102a60 2024-04-26T21:17:57.299 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-26T21:17:57.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f07a40a3c40 msgr2=0x7f07a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f07a40a3c40 0x7f07a40a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005c40 tx=0x7f07ac005bd0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc102a60 msgr2=0x7f07bc197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc102a60 0x7f07bc197f80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f07b000c970 tx=0x7f07b000ce30 comp rx=0 tx=0).stop 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 shutdown_connections 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f07a40a3c40 0x7f07a40a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07bc103420 0x7f07bc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc102a60 0x7f07bc197f80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 --2- 172.21.15.129:0/1841223112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07bc101870 0x7f07bc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 >> 172.21.15.129:0/1841223112 conn(0x7f07bc0fcde0 msgr2=0x7f07bc105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 shutdown_connections 2024-04-26T21:17:57.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.300+0000 7f07c196a640 1 -- 172.21.15.129:0/1841223112 wait complete. 2024-04-26T21:17:57.313 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:17:57.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.410+0000 7fcfb5664640 1 -- 172.21.15.129:0/3520541645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0103420 msgr2=0x7fcfb0109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.410+0000 7fcfb5664640 1 --2- 172.21.15.129:0/3520541645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0103420 0x7fcfb0109cb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009900 tx=0x7fcfa802a410 comp rx=0 tx=0).stop 2024-04-26T21:17:57.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.410+0000 7fcfb5664640 1 -- 172.21.15.129:0/3520541645 shutdown_connections 2024-04-26T21:17:57.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.410+0000 7fcfb5664640 1 --2- 172.21.15.129:0/3520541645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb0102a60 0x7fcfb0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.410+0000 7fcfb5664640 1 --2- 172.21.15.129:0/3520541645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0103420 0x7fcfb0109cb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.410+0000 7fcfb5664640 1 --2- 172.21.15.129:0/3520541645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0101870 0x7fcfb0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.410+0000 7fcfb5664640 1 -- 172.21.15.129:0/3520541645 >> 172.21.15.129:0/3520541645 conn(0x7fcfb00fcde0 msgr2=0x7fcfb00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:57.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.410+0000 7fcfb5664640 1 -- 172.21.15.129:0/3520541645 shutdown_connections 2024-04-26T21:17:57.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.410+0000 7fcfb5664640 1 -- 172.21.15.129:0/3520541645 wait complete. 2024-04-26T21:17:57.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfb5664640 1 Processor -- start 2024-04-26T21:17:57.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfb5664640 1 -- start start 2024-04-26T21:17:57.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfb5664640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0101870 0x7fcfb0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfb5664640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0102a60 0x7fcfb0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfb5664640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb0103420 0x7fcfb019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfb5664640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb010d380 con 0x7fcfb0101870 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfb5664640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb010d200 con 0x7fcfb0103420 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfb5664640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb010d500 con 0x7fcfb0102a60 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfaeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0101870 0x7fcfb0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfaeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0101870 0x7fcfb0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33330/0 (socket says 172.21.15.129:33330) 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.411+0000 7fcfaeffd640 1 -- 172.21.15.129:0/987942655 learned_addr learned my addr 172.21.15.129:0/987942655 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfae7fc640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0102a60 0x7fcfb0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfaf7fe640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb0103420 0x7fcfb019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfaeffd640 1 -- 172.21.15.129:0/987942655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0102a60 msgr2=0x7fcfb0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfaeffd640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0102a60 0x7fcfb0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfaeffd640 1 -- 172.21.15.129:0/987942655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb0103420 msgr2=0x7fcfb019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfaeffd640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb0103420 0x7fcfb019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfaeffd640 1 -- 172.21.15.129:0/987942655 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb019f640 con 0x7fcfb0101870 2024-04-26T21:17:57.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfaeffd640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0101870 0x7fcfb0197a20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009fd0 tx=0x7fcfa000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:57.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcf97fff640 1 -- 172.21.15.129:0/987942655 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0011470 con 0x7fcfb0101870 2024-04-26T21:17:57.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcf97fff640 1 -- 172.21.15.129:0/987942655 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa0011ac0 con 0x7fcfb0101870 2024-04-26T21:17:57.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb019f930 con 0x7fcfb0101870 2024-04-26T21:17:57.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcf97fff640 1 -- 172.21.15.129:0/987942655 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa001aab0 con 0x7fcfb0101870 2024-04-26T21:17:57.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.412+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb019fe80 con 0x7fcfb0101870 2024-04-26T21:17:57.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.413+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb0108c10 con 0x7fcfb0101870 2024-04-26T21:17:57.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.414+0000 7fcf97fff640 1 -- 172.21.15.129:0/987942655 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fcfa001ac50 con 0x7fcfb0101870 2024-04-26T21:17:57.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.414+0000 7fcf97fff640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf980a3c40 0x7fcf980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.415+0000 7fcfae7fc640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf980a3c40 0x7fcf980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.415+0000 7fcf97fff640 1 -- 172.21.15.129:0/987942655 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fcfa00d0b30 con 0x7fcfb0101870 2024-04-26T21:17:57.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.415+0000 7fcfae7fc640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf980a3c40 0x7fcf980a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0198f30 tx=0x7fcfa400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:57.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.419+0000 7fcf97fff640 1 -- 172.21.15.129:0/987942655 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcfa0016030 con 0x7fcfb0101870 2024-04-26T21:17:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:57 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/557023550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:57 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2863001606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:57 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1772011629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:57 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1841223112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:57.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.635+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfb002dad0 con 0x7fcfb0101870 2024-04-26T21:17:57.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.636+0000 7fcf97fff640 1 -- 172.21.15.129:0/987942655 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7fcfb002dad0 con 0x7fcfb0101870 2024-04-26T21:17:57.637 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:57.638 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:17:57.638 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:17:57.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.638+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf980a3c40 msgr2=0x7fcf980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.638+0000 7fcfb5664640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf980a3c40 0x7fcf980a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0198f30 tx=0x7fcfa400b390 comp rx=0 tx=0).stop 2024-04-26T21:17:57.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.638+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0101870 msgr2=0x7fcfb0197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.638+0000 7fcfb5664640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0101870 0x7fcfb0197a20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009fd0 tx=0x7fcfa000c270 comp rx=0 tx=0).stop 2024-04-26T21:17:57.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.638+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 shutdown_connections 2024-04-26T21:17:57.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.638+0000 7fcfb5664640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fcf980a3c40 0x7fcf980a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.639+0000 7fcfb5664640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb0103420 0x7fcfb019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.639+0000 7fcfb5664640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfb0102a60 0x7fcfb0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.639+0000 7fcfb5664640 1 --2- 172.21.15.129:0/987942655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb0101870 0x7fcfb0197a20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.639+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 >> 172.21.15.129:0/987942655 conn(0x7fcfb00fcde0 msgr2=0x7fcfb0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:57.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.639+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 shutdown_connections 2024-04-26T21:17:57.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.639+0000 7fcfb5664640 1 -- 172.21.15.129:0/987942655 wait complete. 2024-04-26T21:17:57.650 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:57 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/557023550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:57.650 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:57 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2863001606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:57.650 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:57 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1772011629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:57.650 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:57 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1841223112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:57.651 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:17:57.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:57 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/557023550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:57.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:57 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2863001606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:57 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1772011629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:57.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:57 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1841223112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.749+0000 7fea9631c640 1 -- 172.21.15.129:0/1303471320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90105d30 msgr2=0x7fea90108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.749+0000 7fea9631c640 1 --2- 172.21.15.129:0/1303471320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90105d30 0x7fea90108120 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea80009900 tx=0x7fea8002a410 comp rx=0 tx=0).stop 2024-04-26T21:17:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.749+0000 7fea9631c640 1 -- 172.21.15.129:0/1303471320 shutdown_connections 2024-04-26T21:17:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.749+0000 7fea9631c640 1 --2- 172.21.15.129:0/1303471320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea90108660 0x7fea9010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.749+0000 7fea9631c640 1 --2- 172.21.15.129:0/1303471320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90105d30 0x7fea90108120 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.749+0000 7fea9631c640 1 --2- 172.21.15.129:0/1303471320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea90103400 0x7fea901057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.749+0000 7fea9631c640 1 -- 172.21.15.129:0/1303471320 >> 172.21.15.129:0/1303471320 conn(0x7fea900fce10 msgr2=0x7fea900ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.749+0000 7fea9631c640 1 -- 172.21.15.129:0/1303471320 shutdown_connections 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.749+0000 7fea9631c640 1 -- 172.21.15.129:0/1303471320 wait complete. 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea9631c640 1 Processor -- start 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea9631c640 1 -- start start 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea9631c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea90103400 0x7fea90198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea9631c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea90105d30 0x7fea90199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea9631c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90108660 0x7fea901a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea9631c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fea9010e4f0 con 0x7fea90108660 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea9631c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea9010e370 con 0x7fea90105d30 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea9631c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea9010e670 con 0x7fea90103400 2024-04-26T21:17:57.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea948b9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90108660 0x7fea901a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea948b9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90108660 0x7fea901a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33356/0 (socket says 172.21.15.129:33356) 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea948b9640 1 -- 172.21.15.129:0/2881824243 learned_addr learned my addr 172.21.15.129:0/2881824243 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea8ffff640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea90103400 0x7fea90198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea8f7fe640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea90105d30 0x7fea90199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea948b9640 1 -- 172.21.15.129:0/2881824243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea90103400 msgr2=0x7fea90198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea948b9640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea90103400 0x7fea90198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea948b9640 1 -- 172.21.15.129:0/2881824243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea90105d30 msgr2=0x7fea90199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea948b9640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea90105d30 0x7fea90199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.750+0000 7fea948b9640 1 -- 172.21.15.129:0/2881824243 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea901a07e0 con 0x7fea90108660 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.751+0000 7fea948b9640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90108660 0x7fea901a0170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fea8800eaa0 tx=0x7fea8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.751+0000 7fea8d7fa640 1 -- 172.21.15.129:0/2881824243 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8800cdb0 con 0x7fea90108660 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.751+0000 7fea8d7fa640 1 -- 172.21.15.129:0/2881824243 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea88005020 con 0x7fea90108660 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.751+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea901a0ad0 con 0x7fea90108660 2024-04-26T21:17:57.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.751+0000 7fea8d7fa640 1 -- 172.21.15.129:0/2881824243 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880116d0 con 0x7fea90108660 2024-04-26T21:17:57.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.751+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901a0fa0 con 0x7fea90108660 2024-04-26T21:17:57.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.752+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea900fe900 con 0x7fea90108660 2024-04-26T21:17:57.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.753+0000 7fea8d7fa640 1 -- 172.21.15.129:0/2881824243 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fea880051d0 con 0x7fea90108660 2024-04-26T21:17:57.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.753+0000 7fea8d7fa640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fea780a3c40 0x7fea780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:57.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.754+0000 7fea8ffff640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fea780a3c40 0x7fea780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:57.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.754+0000 7fea8d7fa640 1 -- 172.21.15.129:0/2881824243 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fea880d06f0 con 0x7fea90108660 2024-04-26T21:17:57.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.754+0000 7fea8ffff640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fea780a3c40 0x7fea780a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea84009bd0 tx=0x7fea84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:57.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.758+0000 7fea8d7fa640 1 -- 172.21.15.129:0/2881824243 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fea88016030 con 0x7fea90108660 2024-04-26T21:17:57.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.937+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea90063530 con 0x7fea90108660 2024-04-26T21:17:57.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.939+0000 7fea8d7fa640 1 -- 172.21.15.129:0/2881824243 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7fea8809c910 con 0x7fea90108660 2024-04-26T21:17:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:57.940 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:17:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fea780a3c40 msgr2=0x7fea780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fea780a3c40 0x7fea780a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea84009bd0 tx=0x7fea84005f90 comp rx=0 tx=0).stop 2024-04-26T21:17:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90108660 msgr2=0x7fea901a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90108660 0x7fea901a0170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fea8800eaa0 tx=0x7fea8800ef60 comp rx=0 tx=0).stop 2024-04-26T21:17:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 shutdown_connections 2024-04-26T21:17:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fea780a3c40 0x7fea780a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea90105d30 0x7fea90199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea90108660 0x7fea901a0170 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 --2- 172.21.15.129:0/2881824243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea90103400 0x7fea90198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 >> 172.21.15.129:0/2881824243 conn(0x7fea900fce10 msgr2=0x7fea90103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 shutdown_connections 2024-04-26T21:17:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:57.941+0000 7fea9631c640 1 -- 172.21.15.129:0/2881824243 wait complete. 2024-04-26T21:17:57.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:16:01.621500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:16:02.592849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:17:57.954 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:17:58.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.053+0000 7f737354e640 1 -- 172.21.15.129:0/395463386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c104530 msgr2=0x7f736c10adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.053+0000 7f737354e640 1 --2- 172.21.15.129:0/395463386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c104530 0x7f736c10adc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7364009ca0 tx=0x7f736402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.053+0000 7f737354e640 1 -- 172.21.15.129:0/395463386 shutdown_connections 2024-04-26T21:17:58.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.053+0000 7f737354e640 1 --2- 172.21.15.129:0/395463386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f736c103b70 0x7f736c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.053+0000 7f737354e640 1 --2- 172.21.15.129:0/395463386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c104530 0x7f736c10adc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.053+0000 7f737354e640 1 --2- 172.21.15.129:0/395463386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c102980 0x7f736c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.053+0000 7f737354e640 1 -- 172.21.15.129:0/395463386 >> 172.21.15.129:0/395463386 conn(0x7f736c0fdf10 msgr2=0x7f736c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:58.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 -- 172.21.15.129:0/395463386 shutdown_connections 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 -- 172.21.15.129:0/395463386 wait complete. 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 Processor -- start 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 -- start start 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c102980 0x7f736c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c103b70 0x7f736c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f736c104530 0x7f736c1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f736c10e490 con 0x7f736c103b70 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f736c10e310 con 0x7f736c104530 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f737354e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f736c10e610 con 0x7f736c102980 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f7370ae9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c103b70 0x7f736c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f7370ae9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c103b70 0x7f736c1990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33368/0 (socket says 172.21.15.129:33368) 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f7370ae9640 1 -- 172.21.15.129:0/847012431 learned_addr learned my addr 172.21.15.129:0/847012431 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:58.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f73712ea640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c102980 0x7f736c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.054+0000 7f7371aeb640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f736c104530 0x7f736c1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f7370ae9640 1 -- 172.21.15.129:0/847012431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c102980 msgr2=0x7f736c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f7370ae9640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c102980 0x7f736c198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f7370ae9640 1 -- 172.21.15.129:0/847012431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f736c104530 msgr2=0x7f736c1a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f7370ae9640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f736c104530 0x7f736c1a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f7370ae9640 1 -- 172.21.15.129:0/847012431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c1a0780 con 0x7f736c103b70 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f7370ae9640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c103b70 0x7f736c1990a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f735c00d950 tx=0x7f735c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f73627fc640 1 -- 172.21.15.129:0/847012431 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c017070 con 0x7f736c103b70 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f73627fc640 1 -- 172.21.15.129:0/847012431 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f735c009c30 con 0x7f736c103b70 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736c1a0a70 con 0x7f736c103b70 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f73627fc640 1 -- 172.21.15.129:0/847012431 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c00f620 con 0x7f736c103b70 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.055+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736c1a0fc0 con 0x7f736c103b70 2024-04-26T21:17:58.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.056+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f736c109d40 con 0x7f736c103b70 2024-04-26T21:17:58.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.057+0000 7f73627fc640 1 -- 172.21.15.129:0/847012431 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f735c010460 con 0x7f736c103b70 2024-04-26T21:17:58.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.058+0000 7f73627fc640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f73580a3c40 0x7f73580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.058+0000 7f73712ea640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f73580a3c40 0x7f73580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.058+0000 7f73627fc640 1 -- 172.21.15.129:0/847012431 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f735c0d0100 con 0x7f736c103b70 2024-04-26T21:17:58.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.058+0000 7f73712ea640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f73580a3c40 0x7f73580a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f736c1039d0 tx=0x7f736800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:58.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.062+0000 7f73627fc640 1 -- 172.21.15.129:0/847012431 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f735c009de0 con 0x7f736c103b70 2024-04-26T21:17:58.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.239+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f736c02dc60 con 0x7f736c103b70 2024-04-26T21:17:58.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.239+0000 7f73627fc640 1 -- 172.21.15.129:0/847012431 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f735c09c320 con 0x7f736c103b70 2024-04-26T21:17:58.241 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.241+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f73580a3c40 msgr2=0x7f73580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.241+0000 7f737354e640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f73580a3c40 0x7f73580a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f736c1039d0 tx=0x7f736800b390 comp rx=0 tx=0).stop 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.241+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c103b70 msgr2=0x7f736c1990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.241+0000 7f737354e640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c103b70 0x7f736c1990a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f735c00d950 tx=0x7f735c00de10 comp rx=0 tx=0).stop 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.242+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 shutdown_connections 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.242+0000 7f737354e640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f73580a3c40 0x7f73580a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.242+0000 7f737354e640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f736c104530 0x7f736c1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.242+0000 7f737354e640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f736c103b70 0x7f736c1990a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.242+0000 7f737354e640 1 --2- 172.21.15.129:0/847012431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c102980 0x7f736c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.242+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 >> 172.21.15.129:0/847012431 conn(0x7f736c0fdf10 msgr2=0x7f736c106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:58.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.242+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 shutdown_connections 2024-04-26T21:17:58.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.242+0000 7f737354e640 1 -- 172.21.15.129:0/847012431 wait complete. 2024-04-26T21:17:58.254 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:17:58.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.354+0000 7fb7d28c4640 1 -- 172.21.15.129:0/513054159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc102a60 msgr2=0x7fb7cc102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.354+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/513054159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc102a60 0x7fb7cc102ee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009900 tx=0x7fb7c002a410 comp rx=0 tx=0).stop 2024-04-26T21:17:58.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.354+0000 7fb7d28c4640 1 -- 172.21.15.129:0/513054159 shutdown_connections 2024-04-26T21:17:58.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.354+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/513054159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7cc103420 0x7fb7cc109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.354+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/513054159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc102a60 0x7fb7cc102ee0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.354+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/513054159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc101870 0x7fb7cc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.354+0000 7fb7d28c4640 1 -- 172.21.15.129:0/513054159 >> 172.21.15.129:0/513054159 conn(0x7fb7cc0fcde0 msgr2=0x7fb7cc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.354+0000 7fb7d28c4640 1 -- 172.21.15.129:0/513054159 shutdown_connections 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.354+0000 7fb7d28c4640 1 -- 172.21.15.129:0/513054159 wait complete. 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d28c4640 1 Processor -- start 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d28c4640 1 -- start start 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d28c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7cc101870 0x7fb7cc198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d28c4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102a60 0x7fb7cc1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d28c4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc103420 0x7fb7cc1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d28c4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10d410 con 0x7fb7cc103420 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d28c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10d290 con 0x7fb7cc101870 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d28c4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10d590 con 0x7fb7cc102a60 2024-04-26T21:17:58.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d0e61640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc103420 0x7fb7cc1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d0e61640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc103420 0x7fb7cc1a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33386/0 (socket says 172.21.15.129:33386) 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.355+0000 7fb7d0e61640 1 -- 172.21.15.129:0/626695044 learned_addr learned my addr 172.21.15.129:0/626695044 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7cb7fe640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102a60 0x7fb7cc1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7cbfff640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7cc101870 0x7fb7cc198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7d0e61640 1 -- 172.21.15.129:0/626695044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102a60 msgr2=0x7fb7cc1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7d0e61640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102a60 0x7fb7cc1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7d0e61640 1 -- 172.21.15.129:0/626695044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7cc101870 msgr2=0x7fb7cc198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7d0e61640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7cc101870 0x7fb7cc198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7d0e61640 1 -- 172.21.15.129:0/626695044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc1a0870 con 0x7fb7cc103420 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7cb7fe640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102a60 0x7fb7cc1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7cbfff640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7cc101870 0x7fb7cc198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7d0e61640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc103420 0x7fb7cc1a01d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400cce0 tx=0x7fb7c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7c97fa640 1 -- 172.21.15.129:0/626695044 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400f450 con 0x7fb7cc103420 2024-04-26T21:17:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7c97fa640 1 -- 172.21.15.129:0/626695044 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c400faa0 con 0x7fb7cc103420 2024-04-26T21:17:58.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc1a0b60 con 0x7fb7cc103420 2024-04-26T21:17:58.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7c97fa640 1 -- 172.21.15.129:0/626695044 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4018b70 con 0x7fb7cc103420 2024-04-26T21:17:58.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.356+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc1a1060 con 0x7fb7cc103420 2024-04-26T21:17:58.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.357+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7cc1a1290 con 0x7fb7cc103420 2024-04-26T21:17:58.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.358+0000 7fb7c97fa640 1 -- 172.21.15.129:0/626695044 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb7c4018d10 con 0x7fb7cc103420 2024-04-26T21:17:58.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.359+0000 7fb7c97fa640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb7b40a3c40 0x7fb7b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.359+0000 7fb7cbfff640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb7b40a3c40 0x7fb7b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.359+0000 7fb7c97fa640 1 -- 172.21.15.129:0/626695044 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb7c40d0b20 con 0x7fb7cc103420 2024-04-26T21:17:58.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.359+0000 7fb7cbfff640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb7b40a3c40 0x7fb7b40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc006fd0 tx=0x7fb7bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:58.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.363+0000 7fb7c97fa640 1 -- 172.21.15.129:0/626695044 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb7c4014030 con 0x7fb7cc103420 2024-04-26T21:17:58.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.540+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7cc1a1460 con 0x7fb7cc103420 2024-04-26T21:17:58.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.540+0000 7fb7c97fa640 1 -- 172.21.15.129:0/626695044 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fb7c409d390 con 0x7fb7cc103420 2024-04-26T21:17:58.542 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb7b40a3c40 msgr2=0x7fb7b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb7b40a3c40 0x7fb7b40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc006fd0 tx=0x7fb7bc008040 comp rx=0 tx=0).stop 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc103420 msgr2=0x7fb7cc1a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc103420 0x7fb7cc1a01d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400cce0 tx=0x7fb7c400a620 comp rx=0 tx=0).stop 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 shutdown_connections 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb7b40a3c40 0x7fb7b40a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7cc103420 0x7fb7cc1a01d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102a60 0x7fb7cc1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 --2- 172.21.15.129:0/626695044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7cc101870 0x7fb7cc198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.542+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 >> 172.21.15.129:0/626695044 conn(0x7fb7cc0fcde0 msgr2=0x7fb7cc0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.543+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 shutdown_connections 2024-04-26T21:17:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.543+0000 7fb7d28c4640 1 -- 172.21.15.129:0/626695044 wait complete. 2024-04-26T21:17:58.554 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:58 smithi129 ceph-mon[29237]: pgmap v349: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:58.554 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:58 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/987942655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:58.554 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:58 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2881824243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:58.554 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:58 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/847012431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:58.555 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:17:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:58 smithi150 ceph-mon[32290]: pgmap v349: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:58 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/987942655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:58 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2881824243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:58.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:58 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/847012431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:58.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.654+0000 7f875ac45640 1 -- 172.21.15.129:0/1778876669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8754076910 msgr2=0x7f8754074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.654+0000 7f875ac45640 1 --2- 172.21.15.129:0/1778876669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8754076910 0x7f8754074d60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8750009900 tx=0x7f875002a480 comp rx=0 tx=0).stop 2024-04-26T21:17:58.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.654+0000 7f875ac45640 1 -- 172.21.15.129:0/1778876669 shutdown_connections 2024-04-26T21:17:58.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.654+0000 7f875ac45640 1 --2- 172.21.15.129:0/1778876669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8754075330 0x7f8754075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.654+0000 7f875ac45640 1 --2- 172.21.15.129:0/1778876669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f875410a810 0x7f875410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.654+0000 7f875ac45640 1 --2- 172.21.15.129:0/1778876669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8754076910 0x7f8754074d60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.654+0000 7f875ac45640 1 -- 172.21.15.129:0/1778876669 >> 172.21.15.129:0/1778876669 conn(0x7f87540fdf10 msgr2=0x7f8754100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.654+0000 7f875ac45640 1 -- 172.21.15.129:0/1778876669 shutdown_connections 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.654+0000 7f875ac45640 1 -- 172.21.15.129:0/1778876669 wait complete. 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f875ac45640 1 Processor -- start 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f875ac45640 1 -- start start 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f875ac45640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8754075330 0x7f875419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f875ac45640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8754076910 0x7f875419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f875ac45640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f875410a810 0x7f87541a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f875ac45640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87541106a0 con 0x7f875410a810 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f875ac45640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8754110520 con 0x7f8754076910 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f875ac45640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8754110820 con 0x7f8754075330 2024-04-26T21:17:58.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87591e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f875410a810 0x7f87541a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87591e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f875410a810 0x7f87541a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33402/0 (socket says 172.21.15.129:33402) 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87591e2640 1 -- 172.21.15.129:0/769135255 learned_addr learned my addr 172.21.15.129:0/769135255 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87589e1640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8754075330 0x7f875419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f874bfff640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8754076910 0x7f875419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87591e2640 1 -- 172.21.15.129:0/769135255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8754075330 msgr2=0x7f875419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87591e2640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8754075330 0x7f875419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87591e2640 1 -- 172.21.15.129:0/769135255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8754076910 msgr2=0x7f875419b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87591e2640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8754076910 0x7f875419b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87591e2640 1 -- 172.21.15.129:0/769135255 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87541a29f0 con 0x7f875410a810 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.655+0000 7f87591e2640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f875410a810 0x7f87541a2380 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f874c00eb10 tx=0x7f874c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.656+0000 7f8749ffb640 1 -- 172.21.15.129:0/769135255 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c004b20 con 0x7f875410a810 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.656+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87541a2ce0 con 0x7f875410a810 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.656+0000 7f8749ffb640 1 -- 172.21.15.129:0/769135255 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f874c004cc0 con 0x7f875410a810 2024-04-26T21:17:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.656+0000 7f8749ffb640 1 -- 172.21.15.129:0/769135255 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c010690 con 0x7f875410a810 2024-04-26T21:17:58.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.656+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87541a3220 con 0x7f875410a810 2024-04-26T21:17:58.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.657+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87541095d0 con 0x7f875410a810 2024-04-26T21:17:58.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.658+0000 7f8749ffb640 1 -- 172.21.15.129:0/769135255 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f874c005000 con 0x7f875410a810 2024-04-26T21:17:58.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.658+0000 7f8749ffb640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f873c0a3c40 0x7f873c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.659+0000 7f87589e1640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f873c0a3c40 0x7f873c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.659+0000 7f8749ffb640 1 -- 172.21.15.129:0/769135255 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f874c0d0e40 con 0x7f875410a810 2024-04-26T21:17:58.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.659+0000 7f87589e1640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f873c0a3c40 0x7f873c0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8750000c00 tx=0x7f87500314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:58.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.662+0000 7f8749ffb640 1 -- 172.21.15.129:0/769135255 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f874c054060 con 0x7f875410a810 2024-04-26T21:17:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:58 smithi171 ceph-mon[33478]: pgmap v349: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:58 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/987942655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:58 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2881824243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:17:58.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:58 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/847012431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:17:58.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.840+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f875402dad0 con 0x7f875410a810 2024-04-26T21:17:58.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.841+0000 7f8749ffb640 1 -- 172.21.15.129:0/769135255 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f874c09d6b0 con 0x7f875410a810 2024-04-26T21:17:58.842 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-26T21:17:58.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f873c0a3c40 msgr2=0x7f873c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f873c0a3c40 0x7f873c0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8750000c00 tx=0x7f87500314a0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f875410a810 msgr2=0x7f87541a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f875410a810 0x7f87541a2380 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f874c00eb10 tx=0x7f874c00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 shutdown_connections 2024-04-26T21:17:58.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f873c0a3c40 0x7f873c0a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8754076910 0x7f875419b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f875410a810 0x7f87541a2380 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 --2- 172.21.15.129:0/769135255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8754075330 0x7f875419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 >> 172.21.15.129:0/769135255 conn(0x7f87540fdf10 msgr2=0x7f87541067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:58.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 shutdown_connections 2024-04-26T21:17:58.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.843+0000 7f875ac45640 1 -- 172.21.15.129:0/769135255 wait complete. 2024-04-26T21:17:58.855 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:17:58.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 -- 172.21.15.129:0/400400098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 msgr2=0x7f5b8c102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/400400098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 0x7f5b8c102f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009900 tx=0x7f5b7802a410 comp rx=0 tx=0).stop 2024-04-26T21:17:58.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 -- 172.21.15.129:0/400400098 shutdown_connections 2024-04-26T21:17:58.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/400400098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b8c1034a0 0x7f5b8c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/400400098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 0x7f5b8c102f60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/400400098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b8c1018f0 0x7f5b8c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 -- 172.21.15.129:0/400400098 >> 172.21.15.129:0/400400098 conn(0x7f5b8c0fce80 msgr2=0x7f5b8c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 -- 172.21.15.129:0/400400098 shutdown_connections 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 -- 172.21.15.129:0/400400098 wait complete. 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 Processor -- start 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 -- start start 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b8c1018f0 0x7f5b8c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 0x7f5b8c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b8c1034a0 0x7f5b8c1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c10d390 con 0x7f5b8c102ae0 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c10d210 con 0x7f5b8c1034a0 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b92d0a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c10d510 con 0x7f5b8c1018f0 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b8bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 0x7f5b8c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b8bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 0x7f5b8c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33422/0 (socket says 172.21.15.129:33422) 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b8bfff640 1 -- 172.21.15.129:0/1598918273 learned_addr learned my addr 172.21.15.129:0/1598918273 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:58.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.952+0000 7f5b90aa6640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b8c1018f0 0x7f5b8c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b912a7640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b8c1034a0 0x7f5b8c1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b8bfff640 1 -- 172.21.15.129:0/1598918273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b8c1018f0 msgr2=0x7f5b8c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b8bfff640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b8c1018f0 0x7f5b8c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b8bfff640 1 -- 172.21.15.129:0/1598918273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b8c1034a0 msgr2=0x7f5b8c1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b8bfff640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b8c1034a0 0x7f5b8c1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b8bfff640 1 -- 172.21.15.129:0/1598918273 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c1a29b0 con 0x7f5b8c102ae0 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b90aa6640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b8c1018f0 0x7f5b8c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b912a7640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b8c1034a0 0x7f5b8c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b8bfff640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 0x7f5b8c19b2e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b78031500 tx=0x7f5b780043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b89ffb640 1 -- 172.21.15.129:0/1598918273 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7803f030 con 0x7f5b8c102ae0 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b89ffb640 1 -- 172.21.15.129:0/1598918273 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b78003800 con 0x7f5b8c102ae0 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c1a2c40 con 0x7f5b8c102ae0 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b89ffb640 1 -- 172.21.15.129:0/1598918273 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78035070 con 0x7f5b8c102ae0 2024-04-26T21:17:58.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.953+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c1a3180 con 0x7f5b8c102ae0 2024-04-26T21:17:58.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.954+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c1a3350 con 0x7f5b8c102ae0 2024-04-26T21:17:58.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.955+0000 7f5b89ffb640 1 -- 172.21.15.129:0/1598918273 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5b780039b0 con 0x7f5b8c102ae0 2024-04-26T21:17:58.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.956+0000 7f5b89ffb640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b7c0a3cc0 0x7f5b7c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:58.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.956+0000 7f5b90aa6640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b7c0a3cc0 0x7f5b7c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:58.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.956+0000 7f5b89ffb640 1 -- 172.21.15.129:0/1598918273 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5b780f0fe0 con 0x7f5b8c102ae0 2024-04-26T21:17:58.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.956+0000 7f5b90aa6640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b7c0a3cc0 0x7f5b7c0a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b800058c0 tx=0x7f5b80005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:58.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:58.960+0000 7f5b89ffb640 1 -- 172.21.15.129:0/1598918273 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5b780bd8f0 con 0x7f5b8c102ae0 2024-04-26T21:17:59.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.175+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b8c1a3520 con 0x7f5b8c102ae0 2024-04-26T21:17:59.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.176+0000 7f5b89ffb640 1 -- 172.21.15.129:0/1598918273 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f5b780bd1f0 con 0x7f5b8c102ae0 2024-04-26T21:17:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:59.177 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:17:59.177 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:17:59.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b7c0a3cc0 msgr2=0x7f5b7c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b7c0a3cc0 0x7f5b7c0a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b800058c0 tx=0x7f5b80005850 comp rx=0 tx=0).stop 2024-04-26T21:17:59.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 msgr2=0x7f5b8c19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 0x7f5b8c19b2e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b78031500 tx=0x7f5b780043c0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 shutdown_connections 2024-04-26T21:17:59.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b7c0a3cc0 0x7f5b7c0a6180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b8c1034a0 0x7f5b8c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b8c102ae0 0x7f5b8c19b2e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 --2- 172.21.15.129:0/1598918273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b8c1018f0 0x7f5b8c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 >> 172.21.15.129:0/1598918273 conn(0x7f5b8c0fce80 msgr2=0x7f5b8c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:59.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 shutdown_connections 2024-04-26T21:17:59.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.178+0000 7f5b92d0a640 1 -- 172.21.15.129:0/1598918273 wait complete. 2024-04-26T21:17:59.191 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:17:59.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 -- 172.21.15.129:0/2172043847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4102980 msgr2=0x7f31a4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 --2- 172.21.15.129:0/2172043847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4102980 0x7f31a4102da0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3194009900 tx=0x7f319402a410 comp rx=0 tx=0).stop 2024-04-26T21:17:59.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 -- 172.21.15.129:0/2172043847 shutdown_connections 2024-04-26T21:17:59.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 --2- 172.21.15.129:0/2172043847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a4103b70 0x7f31a4103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 --2- 172.21.15.129:0/2172043847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a4104530 0x7f31a410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 --2- 172.21.15.129:0/2172043847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4102980 0x7f31a4102da0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 -- 172.21.15.129:0/2172043847 >> 172.21.15.129:0/2172043847 conn(0x7f31a40fdf10 msgr2=0x7f31a4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:59.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 -- 172.21.15.129:0/2172043847 shutdown_connections 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 -- 172.21.15.129:0/2172043847 wait complete. 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.288+0000 7f31aa009640 1 Processor -- start 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31aa009640 1 -- start start 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31aa009640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a4102980 0x7f31a419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31aa009640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a4103b70 0x7f31a419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31aa009640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4104530 0x7f31a41a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31aa009640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31a410e4b0 con 0x7f31a4104530 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31aa009640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31a410e330 con 0x7f31a4102980 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31aa009640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f31a410e630 con 0x7f31a4103b70 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4104530 0x7f31a41a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4104530 0x7f31a41a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33442/0 (socket says 172.21.15.129:33442) 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a3fff640 1 -- 172.21.15.129:0/956809281 learned_addr learned my addr 172.21.15.129:0/956809281 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a37fe640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a4102980 0x7f31a419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a2ffd640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a4103b70 0x7f31a419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a3fff640 1 -- 172.21.15.129:0/956809281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a4103b70 msgr2=0x7f31a419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a3fff640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a4103b70 0x7f31a419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a3fff640 1 -- 172.21.15.129:0/956809281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a4102980 msgr2=0x7f31a419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a3fff640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a4102980 0x7f31a419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a3fff640 1 -- 172.21.15.129:0/956809281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a41a2a10 con 0x7f31a4104530 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a3fff640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4104530 0x7f31a41a23a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f319c00eaa0 tx=0x7f319c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a0ff9640 1 -- 172.21.15.129:0/956809281 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c00cdb0 con 0x7f31a4104530 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31a0ff9640 1 -- 172.21.15.129:0/956809281 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f319c005020 con 0x7f31a4104530 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.289+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a41a2d00 con 0x7f31a4104530 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.290+0000 7f31a0ff9640 1 -- 172.21.15.129:0/956809281 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c0116d0 con 0x7f31a4104530 2024-04-26T21:17:59.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.290+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a41a31f0 con 0x7f31a4104530 2024-04-26T21:17:59.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.290+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a41a3480 con 0x7f31a4104530 2024-04-26T21:17:59.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.291+0000 7f31a0ff9640 1 -- 172.21.15.129:0/956809281 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f319c0051d0 con 0x7f31a4104530 2024-04-26T21:17:59.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.292+0000 7f31a0ff9640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f318c0a3c40 0x7f318c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.292+0000 7f31a37fe640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f318c0a3c40 0x7f318c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.292+0000 7f31a0ff9640 1 -- 172.21.15.129:0/956809281 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f319c0d06d0 con 0x7f31a4104530 2024-04-26T21:17:59.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.293+0000 7f31a37fe640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f318c0a3c40 0x7f318c0a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3194009230 tx=0x7f31940314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:59.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.296+0000 7f31a0ff9640 1 -- 172.21.15.129:0/956809281 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f319c016030 con 0x7f31a4104530 2024-04-26T21:17:59.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:59 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/626695044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:59.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:59 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/769135255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:59.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:17:59 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1598918273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:59.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.477+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31a41a37e0 con 0x7f31a4104530 2024-04-26T21:17:59.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.479+0000 7f31a0ff9640 1 -- 172.21.15.129:0/956809281 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7f319c09c8f0 con 0x7f31a4104530 2024-04-26T21:17:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:17:59.480 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f318c0a3c40 msgr2=0x7f318c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f318c0a3c40 0x7f318c0a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3194009230 tx=0x7f31940314a0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4104530 msgr2=0x7f31a41a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4104530 0x7f31a41a23a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f319c00eaa0 tx=0x7f319c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 shutdown_connections 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f318c0a3c40 0x7f318c0a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a4104530 0x7f31a41a23a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a4103b70 0x7f31a419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 --2- 172.21.15.129:0/956809281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a4102980 0x7f31a419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 >> 172.21.15.129:0/956809281 conn(0x7f31a40fdf10 msgr2=0x7f31a4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:59.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 shutdown_connections 2024-04-26T21:17:59.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.481+0000 7f31aa009640 1 -- 172.21.15.129:0/956809281 wait complete. 2024-04-26T21:17:59.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:16:03.677644+0000', 'modified': '2024-04-26T21:16:08.015827+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24497}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6839/2157967831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 2157967831}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 2157967831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/3984403456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 3984403456}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 3984403456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24497]} 2024-04-26T21:17:59.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:17:59.494 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.591+0000 7fef8542a640 1 -- 172.21.15.129:0/1467343215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 msgr2=0x7fef80109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.591+0000 7fef8542a640 1 --2- 172.21.15.129:0/1467343215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 0x7fef80109cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fef78009ca0 tx=0x7fef7802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.591+0000 7fef8542a640 1 -- 172.21.15.129:0/1467343215 shutdown_connections 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.591+0000 7fef8542a640 1 --2- 172.21.15.129:0/1467343215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 0x7fef80109cc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.591+0000 7fef8542a640 1 --2- 172.21.15.129:0/1467343215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef80102a70 0x7fef80102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.591+0000 7fef8542a640 1 --2- 172.21.15.129:0/1467343215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef80101880 0x7fef80101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 -- 172.21.15.129:0/1467343215 >> 172.21.15.129:0/1467343215 conn(0x7fef800fcdf0 msgr2=0x7fef800ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 -- 172.21.15.129:0/1467343215 shutdown_connections 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 -- 172.21.15.129:0/1467343215 wait complete. 2024-04-26T21:17:59.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 Processor -- start 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 -- start start 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef80101880 0x7fef80198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef80102a70 0x7fef801990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 0x7fef801a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef8010e490 con 0x7fef80103430 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef8010e310 con 0x7fef80101880 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.592+0000 7fef8542a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fef8010e610 con 0x7fef80102a70 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 0x7fef801a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 0x7fef801a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33460/0 (socket says 172.21.15.129:33460) 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7f7fe640 1 -- 172.21.15.129:0/4134993062 learned_addr learned my addr 172.21.15.129:0/4134993062 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef80102a70 0x7fef801990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7effd640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef80101880 0x7fef80198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7f7fe640 1 -- 172.21.15.129:0/4134993062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef80102a70 msgr2=0x7fef801990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7f7fe640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef80102a70 0x7fef801990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7f7fe640 1 -- 172.21.15.129:0/4134993062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef80101880 msgr2=0x7fef80198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7f7fe640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef80101880 0x7fef80198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7f7fe640 1 -- 172.21.15.129:0/4134993062 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef801a0780 con 0x7fef80103430 2024-04-26T21:17:59.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef7f7fe640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 0x7fef801a0110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fef780097b0 tx=0x7fef78003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:59.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef67fff640 1 -- 172.21.15.129:0/4134993062 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78004240 con 0x7fef80103430 2024-04-26T21:17:59.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef8542a640 1 -- 172.21.15.129:0/4134993062 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef801a0a10 con 0x7fef80103430 2024-04-26T21:17:59.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef67fff640 1 -- 172.21.15.129:0/4134993062 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef780317c0 con 0x7fef80103430 2024-04-26T21:17:59.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef67fff640 1 -- 172.21.15.129:0/4134993062 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7802fd80 con 0x7fef80103430 2024-04-26T21:17:59.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.593+0000 7fef8542a640 1 -- 172.21.15.129:0/4134993062 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801a0f60 con 0x7fef80103430 2024-04-26T21:17:59.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef67fff640 1 -- 172.21.15.129:0/4134993062 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fef780043e0 con 0x7fef80103430 2024-04-26T21:17:59.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef88223740 1 Processor -- start 2024-04-26T21:17:59.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef88223740 1 -- start start 2024-04-26T21:17:59.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef88223740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f7690 0x55e5d06f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef88223740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d06fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef88223740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d06ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef88223740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e5d065c740 con 0x55e5d06f8080 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef88223740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e5d05f83a0 con 0x55e5d06f7690 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef88223740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e5d0658700 con 0x55e5d06fd1f0 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef7effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d06fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef7effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d06fcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33472/0 (socket says 172.21.15.129:33472) 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef7effd640 1 -- 172.21.15.129:0/1517705482 learned_addr learned my addr 172.21.15.129:0/1517705482 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d06ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef67fff640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef680a3c40 0x7fef680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f7690 0x55e5d06f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef7effd640 1 -- 172.21.15.129:0/1517705482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 msgr2=0x55e5d06ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef7effd640 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d06ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef7effd640 1 -- 172.21.15.129:0/1517705482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f7690 msgr2=0x55e5d06f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:59.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.596+0000 7fef7effd640 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f7690 0x55e5d06f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef7effd640 1 -- 172.21.15.129:0/1517705482 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d0614750 con 0x55e5d06f8080 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f7690 0x55e5d06f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef7f7fe640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef680a3c40 0x7fef680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef7effd640 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d06fcb80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fef7400bd10 tx=0x7fef7400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfce13a785a3ca90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef67fff640 1 -- 172.21.15.129:0/4134993062 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fef780f1df0 con 0x7fef80103430 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef657fa640 1 -- 172.21.15.129:0/1517705482 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74015070 con 0x55e5d06f8080 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef657fa640 1 -- 172.21.15.129:0/1517705482 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7401a910 con 0x55e5d06f8080 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef657fa640 1 -- 172.21.15.129:0/1517705482 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74011870 con 0x55e5d06f8080 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef88223740 1 -- 172.21.15.129:0/1517705482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 msgr2=0x55e5d06fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef7f7fe640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef680a3c40 0x7fef680a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef780049d0 tx=0x7fef7808fbc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.597+0000 7fef88223740 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d06fcb80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fef7400bd10 tx=0x7fef7400f730 comp rx=0 tx=0).stop 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 -- 172.21.15.129:0/1517705482 shutdown_connections 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d06ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d06fcb80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 --2- 172.21.15.129:0/1517705482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f7690 0x55e5d06f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 -- 172.21.15.129:0/1517705482 >> 172.21.15.129:0/1517705482 conn(0x55e5d06f2d00 msgr2=0x55e5d06f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 -- 172.21.15.129:0/1517705482 shutdown_connections 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 -- 172.21.15.129:0/1517705482 wait complete. 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 -- start start 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 Processor -- start 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06f7690 0x55e5d0701b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d0702040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06fd1f0 0x55e5d0709080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e5d07001a0 con 0x55e5d06f8080 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e5d05e3fe0 con 0x55e5d06fd1f0 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e5d0700320 con 0x55e5d06f7690 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d0702040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef88223740 10 client.0 _ll_register_callbacks cb 0x55e5d05fa150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:17:59.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d0702040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33486/0 (socket says 172.21.15.129:33486) 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7effd640 1 -- 172.21.15.129:0/841176648 learned_addr learned my addr 172.21.15.129:0/841176648 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06f7690 0x55e5d0701b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06fd1f0 0x55e5d0709080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7effd640 1 -- 172.21.15.129:0/841176648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06f7690 msgr2=0x55e5d0701b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7effd640 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06f7690 0x55e5d0701b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7effd640 1 -- 172.21.15.129:0/841176648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06fd1f0 msgr2=0x55e5d0709080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7effd640 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06fd1f0 0x55e5d0709080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.598+0000 7fef7effd640 1 -- 172.21.15.129:0/841176648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d036e2d0 con 0x55e5d06f8080 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef7effd640 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d0702040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fef7400bce0 tx=0x7fef7400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974b78e5dab10b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef62ff5640 1 -- 172.21.15.129:0/841176648 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74028060 con 0x55e5d06f8080 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef62ff5640 1 -- 172.21.15.129:0/841176648 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fef7401acd0 con 0x55e5d06f8080 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef88223740 1 -- 172.21.15.129:0/841176648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7401a910 con 0x55e5d06f8080 2024-04-26T21:17:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef62ff5640 1 -- 172.21.15.129:0/841176648 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fef74015070 con 0x55e5d06f8080 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef88223740 10 client.15219 fetch_fsmap learned FSMap version 19 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef88223740 1 -- 172.21.15.129:0/841176648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e5d04386b0 con 0x55e5d06f8080 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef62ff5640 1 -- 172.21.15.129:0/841176648 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7401a910 con 0x55e5d06f8080 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef62ff5640 1 -- 172.21.15.129:0/841176648 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fef74004b80 con 0x55e5d06f8080 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef88223740 10 client.15219 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef88223740 10 client.15219 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef88223740 20 client.15219 populate_metadata read hostname 'smithi129' 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef88223740 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5d070e040 0x55e5d070f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef88223740 4 client.15219 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.599+0000 7fef88223740 1 -- 172.21.15.129:0/841176648 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5d066a350 con 0x55e5d070e040 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.600+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5d070e040 0x55e5d070f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.600+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5d070e040 0x55e5d070f4b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674d971690bfe236 server_cookie=276cf3e5a64d9c94 in_seq=0 out_seq=0 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.600+0000 7fef62ff5640 10 client.15219 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef62ff5640 1 -- 172.21.15.129:0/841176648 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5d066a350 con 0x55e5d070e040 2024-04-26T21:17:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef62ff5640 10 client.15219 handle_command_reply: tid=0 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 client.15219 shutdown 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 -- 172.21.15.129:0/841176648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 msgr2=0x55e5d0702040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d0702040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fef7400bce0 tx=0x7fef7400bfd0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 -- 172.21.15.129:0/841176648 shutdown_connections 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5d070e040 0x55e5d070f4b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06fd1f0 0x55e5d0709080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f8080 0x55e5d0702040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 --2- 172.21.15.129:0/841176648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06f7690 0x55e5d0701b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef62ff5640 0 client.15219 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 -- 172.21.15.129:0/841176648 >> 172.21.15.129:0/841176648 conn(0x55e5d06f2d00 msgr2=0x55e5d06f1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 -- 172.21.15.129:0/841176648 shutdown_connections 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 1 -- 172.21.15.129:0/841176648 wait complete. 2024-04-26T21:17:59.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.601+0000 7fef88223740 20 client.15219 trim_cache size 0 max 16384 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef88223740 1 Processor -- start 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef88223740 1 -- start start 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef88223740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d06f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef88223740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d0709040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef88223740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d06f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef88223740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e5d070d7f0 con 0x55e5d06f7690 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef88223740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e5d070dec0 con 0x55e5d06f8080 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef88223740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e5d070e590 con 0x55e5d06fd1f0 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d06f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d06f94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33494/0 (socket says 172.21.15.129:33494) 2024-04-26T21:17:59.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7e7fc640 1 -- 172.21.15.129:0/3915548830 learned_addr learned my addr 172.21.15.129:0/3915548830 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7effd640 1 --2- 172.21.15.129:0/3915548830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d0709040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7e7fc640 1 -- 172.21.15.129:0/3915548830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 msgr2=0x55e5d06f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/3915548830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d06f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7e7fc640 1 -- 172.21.15.129:0/3915548830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 msgr2=0x55e5d0709040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/3915548830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d0709040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7e7fc640 1 -- 172.21.15.129:0/3915548830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d0702c30 con 0x55e5d06f7690 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7effd640 1 --2- 172.21.15.129:0/3915548830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d0709040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/3915548830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d06f94e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef70005fd0 tx=0x7fef70011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9c63386bff635 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef64ff9640 1 -- 172.21.15.129:0/3915548830 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70004020 con 0x55e5d06f7690 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef64ff9640 1 -- 172.21.15.129:0/3915548830 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef700054d0 con 0x55e5d06f7690 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.619+0000 7fef64ff9640 1 -- 172.21.15.129:0/3915548830 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70004c60 con 0x55e5d06f7690 2024-04-26T21:17:59.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 -- 172.21.15.129:0/3915548830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 msgr2=0x55e5d06f94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 --2- 172.21.15.129:0/3915548830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d06f94e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef70005fd0 tx=0x7fef70011810 comp rx=0 tx=0).stop 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 -- 172.21.15.129:0/3915548830 shutdown_connections 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 --2- 172.21.15.129:0/3915548830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d0709040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 --2- 172.21.15.129:0/3915548830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d06f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 --2- 172.21.15.129:0/3915548830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d06f94e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 -- 172.21.15.129:0/3915548830 >> 172.21.15.129:0/3915548830 conn(0x55e5d06f2d00 msgr2=0x55e5d06f30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 -- 172.21.15.129:0/3915548830 shutdown_connections 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 -- 172.21.15.129:0/3915548830 wait complete. 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 -- start start 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 Processor -- start 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d0705270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d07057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d0708640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e5d070f330 con 0x55e5d06f7690 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e5d070ec60 con 0x55e5d06f8080 2024-04-26T21:17:59.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e5d070fa00 con 0x55e5d06fd1f0 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.620+0000 7fef88223740 10 client.0 _ll_register_callbacks cb 0x7fef7400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d0705270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d0705270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33504/0 (socket says 172.21.15.129:33504) 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 -- 172.21.15.129:0/2138291174 learned_addr learned my addr 172.21.15.129:0/2138291174 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7effd640 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d07057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d0708640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 -- 172.21.15.129:0/2138291174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 msgr2=0x55e5d0708640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d0708640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 -- 172.21.15.129:0/2138291174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 msgr2=0x55e5d07057b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d07057b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 -- 172.21.15.129:0/2138291174 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d0708c80 con 0x55e5d06f7690 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7effd640 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d07057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef7e7fc640 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d0705270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fef70011f50 tx=0x7fef7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e00eb6184c203f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef62ff5640 1 -- 172.21.15.129:0/2138291174 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70034060 con 0x55e5d06f7690 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef62ff5640 1 -- 172.21.15.129:0/2138291174 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fef700201d0 con 0x55e5d06f7690 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef88223740 1 -- 172.21.15.129:0/2138291174 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef700054d0 con 0x55e5d06f7690 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef62ff5640 1 -- 172.21.15.129:0/2138291174 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fef700203d0 con 0x55e5d06f7690 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef88223740 10 client.15225 fetch_fsmap learned FSMap version 19 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef88223740 1 -- 172.21.15.129:0/2138291174 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e5d0774950 con 0x55e5d06f7690 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef62ff5640 1 -- 172.21.15.129:0/2138291174 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef700054d0 con 0x55e5d06f7690 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.621+0000 7fef62ff5640 1 -- 172.21.15.129:0/2138291174 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fef7002c040 con 0x55e5d06f7690 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 10 client.15225 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 20 client.15225 populate_metadata read hostname 'smithi129' 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5d07791a0 0x55e5d0775cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 4 client.15225 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:17:59.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 -- 172.21.15.129:0/2138291174 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "status"}) -- 0x55e5d06641c0 con 0x55e5d07791a0 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef7effd640 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5d07791a0 0x55e5d0775cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef62ff5640 10 client.15225 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef7effd640 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5d07791a0 0x55e5d0775cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257294d0f546a120 server_cookie=86c9664d05934f0f in_seq=0 out_seq=0 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef62ff5640 1 -- 172.21.15.129:0/2138291174 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e5d06641c0 con 0x55e5d07791a0 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef62ff5640 10 client.15225 handle_command_reply: tid=0 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 client.15225 shutdown 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 -- 172.21.15.129:0/2138291174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 msgr2=0x55e5d0705270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d0705270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fef70011f50 tx=0x7fef7000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 -- 172.21.15.129:0/2138291174 shutdown_connections 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5d07791a0 0x55e5d0775cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5d06f8080 0x55e5d07057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef62ff5640 0 client.15225 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5d06fd1f0 0x55e5d0708640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 --2- 172.21.15.129:0/2138291174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5d06f7690 0x55e5d0705270 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.622+0000 7fef88223740 1 -- 172.21.15.129:0/2138291174 >> 172.21.15.129:0/2138291174 conn(0x55e5d06f2d00 msgr2=0x55e5d06f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef88223740 1 -- 172.21.15.129:0/2138291174 shutdown_connections 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef88223740 1 -- 172.21.15.129:0/2138291174 wait complete. 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef88223740 20 client.15225 trim_cache size 0 max 16384 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "eccca2b2-0410-11ef-bc93-c7b262605968", 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 0, 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24497, 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-04-26T21:17:59.631 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 114.94583640800001, 2024-04-26T21:17:59.632 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 19, 2024-04-26T21:17:59.632 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 88, 2024-04-26T21:17:59.632 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 86, 2024-04-26T21:17:59.632 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 120.691022319 2024-04-26T21:17:59.632 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:17:59.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 -- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef680a3c40 msgr2=0x7fef680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef680a3c40 0x7fef680a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef780049d0 tx=0x7fef7808fbc0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 -- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 msgr2=0x7fef801a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 0x7fef801a0110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fef780097b0 tx=0x7fef78003c20 comp rx=0 tx=0).stop 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 -- 172.21.15.129:0/4134993062 shutdown_connections 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef680a3c40 0x7fef680a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef80103430 0x7fef801a0110 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef80102a70 0x7fef801990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 --2- 172.21.15.129:0/4134993062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef80101880 0x7fef80198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 -- 172.21.15.129:0/4134993062 >> 172.21.15.129:0/4134993062 conn(0x7fef800fcdf0 msgr2=0x7fef80105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 -- 172.21.15.129:0/4134993062 shutdown_connections 2024-04-26T21:17:59.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:17:59.623+0000 7fef8542a640 1 -- 172.21.15.129:0/4134993062 wait complete. 2024-04-26T21:17:59.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:59 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/626695044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:59.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:59 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/769135255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:59.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:17:59 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1598918273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:59.635 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T21:17:59.648 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T21:17:59.648 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T21:17:59.648 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T21:17:59.648 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T21:17:59.698 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T21:17:59.699 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T21:17:59.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:17:59.699 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-26T21:17:59.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:17:59.714 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-04-26T21:17:59.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:59 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/626695044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:17:59.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:59 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/769135255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:17:59.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:17:59 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1598918273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:17:59.779 INFO:teuthology.orchestra.run.smithi129.stderr:Cannot find device "ceph-brx" 2024-04-26T21:17:59.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:17:59.781 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-26T21:17:59.781 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:17:59.781 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:17:59.781 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T21:18:00.058 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:18:00.448 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:00 smithi129 ceph-mon[29237]: pgmap v350: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:00.448 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:00 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/956809281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:00 smithi150 ceph-mon[32290]: pgmap v350: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:00 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/956809281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:00 smithi171 ceph-mon[33478]: pgmap v350: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:00 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/956809281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 -- 172.21.15.129:0/3843193723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc06b670 msgr2=0x7f42bc06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 --2- 172.21.15.129:0/3843193723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc06b670 0x7f42bc06bad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009990 tx=0x7f42a8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 -- 172.21.15.129:0/3843193723 shutdown_connections 2024-04-26T21:18:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 --2- 172.21.15.129:0/3843193723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42bc105e60 0x7f42bc1062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 --2- 172.21.15.129:0/3843193723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42bc108350 0x7f42bc06b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 --2- 172.21.15.129:0/3843193723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc06b670 0x7f42bc06bad0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 -- 172.21.15.129:0/3843193723 >> 172.21.15.129:0/3843193723 conn(0x7f42bc0792d0 msgr2=0x7f42bc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 -- 172.21.15.129:0/3843193723 shutdown_connections 2024-04-26T21:18:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 -- 172.21.15.129:0/3843193723 wait complete. 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 Processor -- start 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.833+0000 7f42c2f77700 1 -- start start 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c2f77700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42bc06b670 0x7f42bc1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c2f77700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42bc105e60 0x7f42bc1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c2f77700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc108350 0x7f42bc1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c2f77700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc119c90 con 0x7f42bc108350 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c2f77700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc119b10 con 0x7f42bc06b670 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c2f77700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc119e10 con 0x7f42bc105e60 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c1514700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc108350 0x7f42bc1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c1514700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc108350 0x7f42bc1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33518/0 (socket says 172.21.15.129:33518) 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c1514700 1 -- 172.21.15.129:0/2225022900 learned_addr learned my addr 172.21.15.129:0/2225022900 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c0d13700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42bc06b670 0x7f42bc1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42bbfff700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42bc105e60 0x7f42bc1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c1514700 1 -- 172.21.15.129:0/2225022900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42bc105e60 msgr2=0x7f42bc1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c1514700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42bc105e60 0x7f42bc1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c1514700 1 -- 172.21.15.129:0/2225022900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42bc06b670 msgr2=0x7f42bc1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c1514700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42bc06b670 0x7f42bc1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c1514700 1 -- 172.21.15.129:0/2225022900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc1a52b0 con 0x7f42bc108350 2024-04-26T21:18:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.834+0000 7f42c1514700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc108350 0x7f42bc1a4b70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f42b400b970 tx=0x7f42b400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.835+0000 7f42b9ffb700 1 -- 172.21.15.129:0/2225022900 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4019070 con 0x7f42bc108350 2024-04-26T21:18:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.835+0000 7f42b9ffb700 1 -- 172.21.15.129:0/2225022900 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b400cde0 con 0x7f42bc108350 2024-04-26T21:18:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.835+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc1a55a0 con 0x7f42bc108350 2024-04-26T21:18:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.835+0000 7f42b9ffb700 1 -- 172.21.15.129:0/2225022900 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4007370 con 0x7f42bc108350 2024-04-26T21:18:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.835+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc1ace20 con 0x7f42bc108350 2024-04-26T21:18:01.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.836+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42bc06a110 con 0x7f42bc108350 2024-04-26T21:18:01.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.837+0000 7f42b9ffb700 1 -- 172.21.15.129:0/2225022900 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f42b401c070 con 0x7f42bc108350 2024-04-26T21:18:01.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.837+0000 7f42b9ffb700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f42ac0a3cd0 0x7f42ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:01.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.838+0000 7f42c0d13700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f42ac0a3cd0 0x7f42ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:01.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.838+0000 7f42b9ffb700 1 -- 172.21.15.129:0/2225022900 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f42b40cf940 con 0x7f42bc108350 2024-04-26T21:18:01.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.838+0000 7f42c0d13700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f42ac0a3cd0 0x7f42ac0a6190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009a10 tx=0x7f42b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:01.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:01.842+0000 7f42b9ffb700 1 -- 172.21.15.129:0/2225022900 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f42b409c2c0 con 0x7f42bc108350 2024-04-26T21:18:02.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.087+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f42bc1ad050 con 0x7f42bc108350 2024-04-26T21:18:02.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.087+0000 7f42b9ffb700 1 -- 172.21.15.129:0/2225022900 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+73 (secure 0 0 0) 0x7f42b409bbc0 con 0x7f42bc108350 2024-04-26T21:18:02.088 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:18:02.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f42ac0a3cd0 msgr2=0x7f42ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:02.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f42ac0a3cd0 0x7f42ac0a6190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009a10 tx=0x7f42b0009190 comp rx=0 tx=0).stop 2024-04-26T21:18:02.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc108350 msgr2=0x7f42bc1a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:02.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc108350 0x7f42bc1a4b70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f42b400b970 tx=0x7f42b400bc80 comp rx=0 tx=0).stop 2024-04-26T21:18:02.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 shutdown_connections 2024-04-26T21:18:02.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f42ac0a3cd0 0x7f42ac0a6190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:02.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42bc108350 0x7f42bc1a4b70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:02.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42bc06b670 0x7f42bc1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:02.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 --2- 172.21.15.129:0/2225022900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42bc105e60 0x7f42bc1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:02.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 >> 172.21.15.129:0/2225022900 conn(0x7f42bc0792d0 msgr2=0x7f42bc102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:02.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 shutdown_connections 2024-04-26T21:18:02.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:02.090+0000 7f42c2f77700 1 -- 172.21.15.129:0/2225022900 wait complete. 2024-04-26T21:18:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:02 smithi150 ceph-mon[32290]: pgmap v351: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:02 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2225022900' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:18:02.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:02 smithi129 ceph-mon[29237]: pgmap v351: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:02.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:02 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2225022900' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:18:02.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:02 smithi171 ceph-mon[33478]: pgmap v351: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:02.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:02 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2225022900' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:18:02.939 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:18:02.939 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:18:02.940 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi129.front.sepia.ceph.com 2024-04-26T21:18:02.940 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:18:02.940 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:02.940 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:18:02.940 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:18:02.940 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:18:02.940 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:18:02.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:02.940 DEBUG:teuthology.orchestra.run.smithi129:> ip addr 2024-04-26T21:18:02.956 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:18:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:18:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:18:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-04-26T21:18:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:38 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f0 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:39 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f1 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:60 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:61 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.21.15.129/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft 42051sec preferred_lft 42051sec 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::71d5:251f:8718:9c5a/64 scope link noprefixroute 2024-04-26T21:18:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:02.958 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:18:02.958 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:18:02.958 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-26T21:18:02.958 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:18:02.958 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr flush dev ceph-brx 2024-04-26T21:18:02.958 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set ceph-brx up 2024-04-26T21:18:02.958 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:18:02.958 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-26T21:18:03.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:18:03.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:03.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:03.153 DEBUG:teuthology.orchestra.run.smithi129:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:18:03.176 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-04-26T21:18:03.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:03.178 DEBUG:teuthology.orchestra.run.smithi129:> route 2024-04-26T21:18:03.248 INFO:teuthology.orchestra.run.smithi129.stdout:Kernel IP routing table 2024-04-26T21:18:03.248 INFO:teuthology.orchestra.run.smithi129.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:18:03.248 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:18:03.248 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:18:03.248 INFO:teuthology.orchestra.run.smithi129.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:18:03.249 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:18:03.249 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-26T21:18:03.249 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:18:03.249 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:18:03.249 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:18:03.249 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-26T21:18:03.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:18:03.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:03.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:03.505 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-26T21:18:03.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:03.520 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list-id 2024-04-26T21:18:03.575 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:18:03.575 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-26T21:18:03.575 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:18:03.575 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T21:18:03.575 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-26T21:18:03.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:18:03.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:03.690 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T21:18:03.690 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:18:03.690 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-26T21:18:03.690 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T21:18:03.691 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:18:03.691 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T21:18:03.691 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T21:18:03.691 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T21:18:03.691 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-26T21:18:03.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:18:03.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:03.835 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:18:03.835 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-26T21:18:03.835 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set brx.0 up 2024-04-26T21:18:03.835 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:18:03.835 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-26T21:18:03.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:18:03.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:03.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:18:03.911 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:03.925 INFO:teuthology.orchestra.run.smithi129.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:18:03.925 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:18:03.925 DEBUG:teuthology.orchestra.run.smithi129:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:03.980 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T21:18:03.980 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T21:18:03.980 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/9733ba9c-39b7-458b-9c70-cf2aa87c2e0a 2024-04-26T21:18:03.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:03.980 DEBUG:teuthology.orchestra.run.smithi129:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/9733ba9c-39b7-458b-9c70-cf2aa87c2e0a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-26T21:18:04.077 INFO:teuthology.orchestra.run.smithi129.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-26T21:18:04.077 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-04-26T21:18:04.077 INFO:teuthology.orchestra.run.smithi129.stdout:invalid new device string format 2024-04-26T21:18:04.173 INFO:teuthology.orchestra.run.smithi129.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-26T21:18:04.174 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-04-26T21:18:04.174 INFO:teuthology.orchestra.run.smithi129.stdout:invalid new device string format 2024-04-26T21:18:04.174 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: resolved to: "172.21.15.129:6789,172.21.15.150:6789,172.21.15.171:6789" 2024-04-26T21:18:04.174 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.129:6789,172.21.15.150:6789,172.21.15.171:6789:/volumes/qa/sv_1/9733ba9c-39b7-458b-9c70-cf2aa87c2e0a 2024-04-26T21:18:04.174 INFO:teuthology.orchestra.run.smithi129.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=eccca2b2-0410-11ef-bc93-c7b262605968" will pass to kernel 2024-04-26T21:18:04.175 INFO:teuthology.orchestra.run.smithi129.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:18:04.175 INFO:teuthology.orchestra.run.smithi129.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:18:04.175 INFO:teuthology.orchestra.run.smithi129.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:18:04.175 INFO:teuthology.orchestra.run.smithi129.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:18:04.175 INFO:teuthology.orchestra.run.smithi129.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:18:04.176 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:18:04.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:04.177 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:04.201 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:18:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:18:04.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:04.297 INFO:teuthology.orchestra.run.smithi129.stdout:client15234 2024-04-26T21:18:04.298 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:18:04.298 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:04.314 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:18:04.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:18:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:04.411 INFO:teuthology.orchestra.run.smithi129.stdout:client15234 2024-04-26T21:18:04.411 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:18:04.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:18:04.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:04.467 INFO:teuthology.orchestra.run.smithi129.stdout:client15234 2024-04-26T21:18:04.468 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eccca2b2-0410-11ef-bc93-c7b262605968.client15234/status') 2024-04-26T21:18:04.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:18:04.525 INFO:teuthology.orchestra.run.smithi129.stdout:instance: client.15234 (3)192.168.0.1:0/3748665938 2024-04-26T21:18:04.525 INFO:teuthology.orchestra.run.smithi129.stdout:blocklisted: false 2024-04-26T21:18:04.525 INFO:teuthology.orchestra.run.smithi129.stderr:0+1 records in 2024-04-26T21:18:04.525 INFO:teuthology.orchestra.run.smithi129.stderr:0+1 records out 2024-04-26T21:18:04.526 INFO:teuthology.orchestra.run.smithi129.stderr:70 bytes copied, 3.9751e-05 s, 1.8 MB/s 2024-04-26T21:18:04.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:04.533 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T21:18:04.543 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-26T21:18:04.592 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:04.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:04 smithi150 ceph-mon[32290]: pgmap v352: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:04.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.690+0000 7f0f23519640 1 -- 172.21.15.129:0/1605496566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c102a70 msgr2=0x7f0f1c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:04.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.690+0000 7f0f23519640 1 --2- 172.21.15.129:0/1605496566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c102a70 0x7f0f1c102ef0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009900 tx=0x7f0f0c02a410 comp rx=0 tx=0).stop 2024-04-26T21:18:04.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.690+0000 7f0f23519640 1 -- 172.21.15.129:0/1605496566 shutdown_connections 2024-04-26T21:18:04.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.690+0000 7f0f23519640 1 --2- 172.21.15.129:0/1605496566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f1c103430 0x7f0f1c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.690+0000 7f0f23519640 1 --2- 172.21.15.129:0/1605496566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c102a70 0x7f0f1c102ef0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.690+0000 7f0f23519640 1 --2- 172.21.15.129:0/1605496566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c101880 0x7f0f1c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.690+0000 7f0f23519640 1 -- 172.21.15.129:0/1605496566 >> 172.21.15.129:0/1605496566 conn(0x7f0f1c0fce10 msgr2=0x7f0f1c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.690+0000 7f0f23519640 1 -- 172.21.15.129:0/1605496566 shutdown_connections 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.690+0000 7f0f23519640 1 -- 172.21.15.129:0/1605496566 wait complete. 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f23519640 1 Processor -- start 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f23519640 1 -- start start 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f23519640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c101880 0x7f0f1c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f23519640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c102a70 0x7f0f1c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f23519640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f1c103430 0x7f0f1c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f23519640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c10d3b0 con 0x7f0f1c101880 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f23519640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c10d230 con 0x7f0f1c102a70 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f23519640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c10d530 con 0x7f0f1c103430 2024-04-26T21:18:04.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f212b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c101880 0x7f0f1c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f212b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c101880 0x7f0f1c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59716/0 (socket says 172.21.15.129:59716) 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f212b5640 1 -- 172.21.15.129:0/1029495520 learned_addr learned my addr 172.21.15.129:0/1029495520 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f20ab4640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c102a70 0x7f0f1c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f212b5640 1 -- 172.21.15.129:0/1029495520 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f1c103430 msgr2=0x7f0f1c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f212b5640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f1c103430 0x7f0f1c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f212b5640 1 -- 172.21.15.129:0/1029495520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c102a70 msgr2=0x7f0f1c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f212b5640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c102a70 0x7f0f1c199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.691+0000 7f0f212b5640 1 -- 172.21.15.129:0/1029495520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1c1a0800 con 0x7f0f1c101880 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.692+0000 7f0f212b5640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c101880 0x7f0f1c198be0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800d770 tx=0x7f0f1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.692+0000 7f0f127fc640 1 -- 172.21.15.129:0/1029495520 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18017070 con 0x7f0f1c101880 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.692+0000 7f0f127fc640 1 -- 172.21.15.129:0/1029495520 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f18009c30 con 0x7f0f1c101880 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.692+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1c1a0af0 con 0x7f0f1c101880 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.692+0000 7f0f127fc640 1 -- 172.21.15.129:0/1029495520 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18003cc0 con 0x7f0f1c101880 2024-04-26T21:18:04.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.692+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1c1a1010 con 0x7f0f1c101880 2024-04-26T21:18:04.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.693+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1c1a11e0 con 0x7f0f1c101880 2024-04-26T21:18:04.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.694+0000 7f0f127fc640 1 -- 172.21.15.129:0/1029495520 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0f18003e60 con 0x7f0f1c101880 2024-04-26T21:18:04.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.695+0000 7f0f127fc640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0f080a3c40 0x7f0f080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:04.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.695+0000 7f0f20ab4640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0f080a3c40 0x7f0f080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:04.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.695+0000 7f0f127fc640 1 -- 172.21.15.129:0/1029495520 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0f180cff50 con 0x7f0f1c101880 2024-04-26T21:18:04.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.695+0000 7f0f20ab4640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0f080a3c40 0x7f0f080a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c000c00 tx=0x7f0f0c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:04.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.699+0000 7f0f127fc640 1 -- 172.21.15.129:0/1029495520 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0f1809c7c0 con 0x7f0f1c101880 2024-04-26T21:18:04.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:04 smithi129 ceph-mon[29237]: pgmap v352: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:04.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:04 smithi171 ceph-mon[33478]: pgmap v352: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:04.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.878+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f1c1a1570 con 0x7f0f1c101880 2024-04-26T21:18:04.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.880+0000 7f0f127fc640 1 -- 172.21.15.129:0/1029495520 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7f0f18009de0 con 0x7f0f1c101880 2024-04-26T21:18:04.881 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:04.881 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.881+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0f080a3c40 msgr2=0x7f0f080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.881+0000 7f0f23519640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0f080a3c40 0x7f0f080a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c000c00 tx=0x7f0f0c0314a0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.881+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c101880 msgr2=0x7f0f1c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.881+0000 7f0f23519640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c101880 0x7f0f1c198be0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800d770 tx=0x7f0f1800dc30 comp rx=0 tx=0).stop 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.882+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 shutdown_connections 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.882+0000 7f0f23519640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0f080a3c40 0x7f0f080a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.882+0000 7f0f23519640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c102a70 0x7f0f1c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.882+0000 7f0f23519640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f1c103430 0x7f0f1c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.882+0000 7f0f23519640 1 --2- 172.21.15.129:0/1029495520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f1c101880 0x7f0f1c198be0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.882+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 >> 172.21.15.129:0/1029495520 conn(0x7f0f1c0fce10 msgr2=0x7f0f1c06af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.882+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 shutdown_connections 2024-04-26T21:18:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.882+0000 7f0f23519640 1 -- 172.21.15.129:0/1029495520 wait complete. 2024-04-26T21:18:04.894 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:16:01.621500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:16:02.592849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:04.894 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:04.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.994+0000 7f99e8f0a640 1 -- 172.21.15.129:0/3093246104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 msgr2=0x7f99e410adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:04.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.994+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/3093246104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 0x7f99e410adc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009ca0 tx=0x7f99d402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.994+0000 7f99e8f0a640 1 -- 172.21.15.129:0/3093246104 shutdown_connections 2024-04-26T21:18:04.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.994+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/3093246104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4103b70 0x7f99e4103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.994+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/3093246104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 0x7f99e410adc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.994+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/3093246104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e4102980 0x7f99e4102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.994+0000 7f99e8f0a640 1 -- 172.21.15.129:0/3093246104 >> 172.21.15.129:0/3093246104 conn(0x7f99e40fded0 msgr2=0x7f99e4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.994+0000 7f99e8f0a640 1 -- 172.21.15.129:0/3093246104 shutdown_connections 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.994+0000 7f99e8f0a640 1 -- 172.21.15.129:0/3093246104 wait complete. 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e8f0a640 1 Processor -- start 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e8f0a640 1 -- start start 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e8f0a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4102980 0x7f99e4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e8f0a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e4103b70 0x7f99e4199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e8f0a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 0x7f99e41a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e8f0a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410e490 con 0x7f99e4104530 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e8f0a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410e310 con 0x7f99e4102980 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e8f0a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410e610 con 0x7f99e4103b70 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 0x7f99e41a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 0x7f99e41a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59738/0 (socket says 172.21.15.129:59738) 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e2d9d640 1 -- 172.21.15.129:0/1042384355 learned_addr learned my addr 172.21.15.129:0/1042384355 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:04.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e259c640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4102980 0x7f99e4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e1d9b640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e4103b70 0x7f99e4199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e2d9d640 1 -- 172.21.15.129:0/1042384355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e4103b70 msgr2=0x7f99e4199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e2d9d640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e4103b70 0x7f99e4199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e2d9d640 1 -- 172.21.15.129:0/1042384355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4102980 msgr2=0x7f99e4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e2d9d640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4102980 0x7f99e4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e2d9d640 1 -- 172.21.15.129:0/1042384355 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e41a0700 con 0x7f99e4104530 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e1d9b640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e4103b70 0x7f99e4199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.995+0000 7f99e2d9d640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 0x7f99e41a0090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f99d40097b0 tx=0x7f99d4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.996+0000 7f99d37fe640 1 -- 172.21.15.129:0/1042384355 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4003bf0 con 0x7f99e4104530 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.996+0000 7f99d37fe640 1 -- 172.21.15.129:0/1042384355 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99d4003d90 con 0x7f99e4104530 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.996+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e41a0990 con 0x7f99e4104530 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.996+0000 7f99d37fe640 1 -- 172.21.15.129:0/1042384355 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d402fc70 con 0x7f99e4104530 2024-04-26T21:18:04.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.996+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e41a0ee0 con 0x7f99e4104530 2024-04-26T21:18:05.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.997+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e4109c40 con 0x7f99e4104530 2024-04-26T21:18:05.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.998+0000 7f99d37fe640 1 -- 172.21.15.129:0/1042384355 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f99d4004040 con 0x7f99e4104530 2024-04-26T21:18:05.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.998+0000 7f99d37fe640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f99cc0a3c40 0x7f99cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.999+0000 7f99e259c640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f99cc0a3c40 0x7f99cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.999+0000 7f99d37fe640 1 -- 172.21.15.129:0/1042384355 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f99d40f1ad0 con 0x7f99e4104530 2024-04-26T21:18:05.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:04.999+0000 7f99e259c640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f99cc0a3c40 0x7f99cc0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f99e41039d0 tx=0x7f99dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:05.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.003+0000 7f99d37fe640 1 -- 172.21.15.129:0/1042384355 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f99d40be450 con 0x7f99e4104530 2024-04-26T21:18:05.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.182+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99e4063530 con 0x7f99e4104530 2024-04-26T21:18:05.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.182+0000 7f99d37fe640 1 -- 172.21.15.129:0/1042384355 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f99d40bdd50 con 0x7f99e4104530 2024-04-26T21:18:05.183 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:18:05.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f99cc0a3c40 msgr2=0x7f99cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f99cc0a3c40 0x7f99cc0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f99e41039d0 tx=0x7f99dc005f90 comp rx=0 tx=0).stop 2024-04-26T21:18:05.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 msgr2=0x7f99e41a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 0x7f99e41a0090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f99d40097b0 tx=0x7f99d4031840 comp rx=0 tx=0).stop 2024-04-26T21:18:05.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 shutdown_connections 2024-04-26T21:18:05.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f99cc0a3c40 0x7f99cc0a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e4104530 0x7f99e41a0090 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e4103b70 0x7f99e4199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 --2- 172.21.15.129:0/1042384355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4102980 0x7f99e4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 >> 172.21.15.129:0/1042384355 conn(0x7f99e40fded0 msgr2=0x7f99e4106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:05.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 shutdown_connections 2024-04-26T21:18:05.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.184+0000 7f99e8f0a640 1 -- 172.21.15.129:0/1042384355 wait complete. 2024-04-26T21:18:05.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:18:05.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 -- 172.21.15.129:0/3760875119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038102a60 msgr2=0x7f0038102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 --2- 172.21.15.129:0/3760875119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038102a60 0x7f0038102ee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f002c009900 tx=0x7f002c02a410 comp rx=0 tx=0).stop 2024-04-26T21:18:05.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 -- 172.21.15.129:0/3760875119 shutdown_connections 2024-04-26T21:18:05.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 --2- 172.21.15.129:0/3760875119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0038103420 0x7f0038109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 --2- 172.21.15.129:0/3760875119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038102a60 0x7f0038102ee0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 --2- 172.21.15.129:0/3760875119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0038101870 0x7f0038101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 -- 172.21.15.129:0/3760875119 >> 172.21.15.129:0/3760875119 conn(0x7f00380fce20 msgr2=0x7f00380ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 -- 172.21.15.129:0/3760875119 shutdown_connections 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 -- 172.21.15.129:0/3760875119 wait complete. 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.297+0000 7f003d9f7640 1 Processor -- start 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003d9f7640 1 -- start start 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003d9f7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038101870 0x7f0038197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003d9f7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0038102a60 0x7f0038198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003d9f7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0038103420 0x7f003819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003d9f7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f003810d380 con 0x7f0038101870 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003d9f7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f003810d200 con 0x7f0038103420 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003d9f7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f003810d500 con 0x7f0038102a60 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f0036ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038101870 0x7f0038197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f0036ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038101870 0x7f0038197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59766/0 (socket says 172.21.15.129:59766) 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f0036ffd640 1 -- 172.21.15.129:0/4076558752 learned_addr learned my addr 172.21.15.129:0/4076558752 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f00367fc640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0038102a60 0x7f0038198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f00377fe640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0038103420 0x7f003819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f0036ffd640 1 -- 172.21.15.129:0/4076558752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0038102a60 msgr2=0x7f0038198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f0036ffd640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0038102a60 0x7f0038198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f0036ffd640 1 -- 172.21.15.129:0/4076558752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0038103420 msgr2=0x7f003819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f0036ffd640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0038103420 0x7f003819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f0036ffd640 1 -- 172.21.15.129:0/4076558752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003819f6a0 con 0x7f0038101870 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f0036ffd640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038101870 0x7f0038197ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f002800c970 tx=0x7f002800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003c9f5640 1 -- 172.21.15.129:0/4076558752 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028007c10 con 0x7f0038101870 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003c9f5640 1 -- 172.21.15.129:0/4076558752 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0028007db0 con 0x7f0038101870 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003819f990 con 0x7f0038101870 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003c9f5640 1 -- 172.21.15.129:0/4076558752 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002800f870 con 0x7f0038101870 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.298+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003819feb0 con 0x7f0038101870 2024-04-26T21:18:05.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.299+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0038069ba0 con 0x7f0038101870 2024-04-26T21:18:05.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.300+0000 7f003c9f5640 1 -- 172.21.15.129:0/4076558752 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f002800fa10 con 0x7f0038101870 2024-04-26T21:18:05.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.301+0000 7f003c9f5640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00200a3c40 0x7f00200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.301+0000 7f00367fc640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00200a3c40 0x7f00200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.301+0000 7f003c9f5640 1 -- 172.21.15.129:0/4076558752 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f00280d0950 con 0x7f0038101870 2024-04-26T21:18:05.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.301+0000 7f00367fc640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00200a3c40 0x7f00200a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f002c000c00 tx=0x7f002c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:05.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.305+0000 7f003c9f5640 1 -- 172.21.15.129:0/4076558752 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f002809d250 con 0x7f0038101870 2024-04-26T21:18:05.469 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:05 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1029495520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:05.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:05 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1042384355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:18:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.488+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f003802db00 con 0x7f0038101870 2024-04-26T21:18:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.488+0000 7f003c9f5640 1 -- 172.21.15.129:0/4076558752 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f002809cb50 con 0x7f0038101870 2024-04-26T21:18:05.490 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-26T21:18:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00200a3c40 msgr2=0x7f00200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00200a3c40 0x7f00200a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f002c000c00 tx=0x7f002c0314a0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038101870 msgr2=0x7f0038197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038101870 0x7f0038197ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f002800c970 tx=0x7f002800ce30 comp rx=0 tx=0).stop 2024-04-26T21:18:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 shutdown_connections 2024-04-26T21:18:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f00200a3c40 0x7f00200a6100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0038103420 0x7f003819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0038102a60 0x7f0038198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 --2- 172.21.15.129:0/4076558752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0038101870 0x7f0038197ad0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.490+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 >> 172.21.15.129:0/4076558752 conn(0x7f00380fce20 msgr2=0x7f0038105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:05.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.491+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 shutdown_connections 2024-04-26T21:18:05.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.491+0000 7f003d9f7640 1 -- 172.21.15.129:0/4076558752 wait complete. 2024-04-26T21:18:05.503 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:18:05.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 -- 172.21.15.129:0/3190243056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 msgr2=0x7f45fc10ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 --2- 172.21.15.129:0/3190243056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 0x7f45fc10ae60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009cd0 tx=0x7f45f402a8e0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 -- 172.21.15.129:0/3190243056 shutdown_connections 2024-04-26T21:18:05.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 --2- 172.21.15.129:0/3190243056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45fc103c10 0x7f45fc104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 --2- 172.21.15.129:0/3190243056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 0x7f45fc10ae60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 --2- 172.21.15.129:0/3190243056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45fc100c40 0x7f45fc101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 -- 172.21.15.129:0/3190243056 >> 172.21.15.129:0/3190243056 conn(0x7f45fc0fc570 msgr2=0x7f45fc0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 -- 172.21.15.129:0/3190243056 shutdown_connections 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 -- 172.21.15.129:0/3190243056 wait complete. 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 Processor -- start 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.603+0000 7f4600faf640 1 -- start start 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f4600faf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45fc100c40 0x7f45fc198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f4600faf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45fc103c10 0x7f45fc1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f4600faf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 0x7f45fc1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f4600faf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc10e4c0 con 0x7f45fc1045d0 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f4600faf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc10e340 con 0x7f45fc100c40 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f4600faf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc10e640 con 0x7f45fc103c10 2024-04-26T21:18:05.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 0x7f45fc1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 0x7f45fc1a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59784/0 (socket says 172.21.15.129:59784) 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fad9d640 1 -- 172.21.15.129:0/1927772218 learned_addr learned my addr 172.21.15.129:0/1927772218 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fa59c640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45fc100c40 0x7f45fc198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45f3fff640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45fc103c10 0x7f45fc1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fad9d640 1 -- 172.21.15.129:0/1927772218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45fc103c10 msgr2=0x7f45fc1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fad9d640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45fc103c10 0x7f45fc1990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fad9d640 1 -- 172.21.15.129:0/1927772218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45fc100c40 msgr2=0x7f45fc198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fad9d640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45fc100c40 0x7f45fc198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fad9d640 1 -- 172.21.15.129:0/1927772218 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc1a07b0 con 0x7f45fc1045d0 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fa59c640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45fc100c40 0x7f45fc198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45fad9d640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 0x7f45fc1a0140 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f45f40318b0 tx=0x7f45f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45f37fe640 1 -- 172.21.15.129:0/1927772218 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4004250 con 0x7f45fc1045d0 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45f37fe640 1 -- 172.21.15.129:0/1927772218 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f40305a0 con 0x7f45fc1045d0 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc1a0a40 con 0x7f45fc1045d0 2024-04-26T21:18:05.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f45f37fe640 1 -- 172.21.15.129:0/1927772218 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f40466c0 con 0x7f45fc1045d0 2024-04-26T21:18:05.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.604+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc1a0f90 con 0x7f45fc1045d0 2024-04-26T21:18:05.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.605+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45fc109d90 con 0x7f45fc1045d0 2024-04-26T21:18:05.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.606+0000 7f45f37fe640 1 -- 172.21.15.129:0/1927772218 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f45f4030750 con 0x7f45fc1045d0 2024-04-26T21:18:05.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.607+0000 7f45f37fe640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f45e80a3c40 0x7f45e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.607+0000 7f45fa59c640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f45e80a3c40 0x7f45e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.607+0000 7f45f37fe640 1 -- 172.21.15.129:0/1927772218 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f45f402e030 con 0x7f45fc1045d0 2024-04-26T21:18:05.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.607+0000 7f45fa59c640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f45e80a3c40 0x7f45e80a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0058c0 tx=0x7f45ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:05.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.611+0000 7f45f37fe640 1 -- 172.21.15.129:0/1927772218 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f45f404b060 con 0x7f45fc1045d0 2024-04-26T21:18:05.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:05 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1029495520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:05.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:05 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1042384355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:18:05.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.788+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f45fc02dcd0 con 0x7f45fc1045d0 2024-04-26T21:18:05.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.789+0000 7f45f37fe640 1 -- 172.21.15.129:0/1927772218 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f45f40be5f0 con 0x7f45fc1045d0 2024-04-26T21:18:05.790 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-26T21:18:05.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f45e80a3c40 msgr2=0x7f45e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f45e80a3c40 0x7f45e80a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0058c0 tx=0x7f45ec005850 comp rx=0 tx=0).stop 2024-04-26T21:18:05.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 msgr2=0x7f45fc1a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 0x7f45fc1a0140 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f45f40318b0 tx=0x7f45f4003c10 comp rx=0 tx=0).stop 2024-04-26T21:18:05.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 shutdown_connections 2024-04-26T21:18:05.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f45e80a3c40 0x7f45e80a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45fc1045d0 0x7f45fc1a0140 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45fc103c10 0x7f45fc1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 --2- 172.21.15.129:0/1927772218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45fc100c40 0x7f45fc198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 >> 172.21.15.129:0/1927772218 conn(0x7f45fc0fc570 msgr2=0x7f45fc106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:05.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 shutdown_connections 2024-04-26T21:18:05.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.791+0000 7f4600faf640 1 -- 172.21.15.129:0/1927772218 wait complete. 2024-04-26T21:18:05.804 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:05.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:05 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1029495520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:05.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:05 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1042384355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:18:05.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 -- 172.21.15.129:0/2156587076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 msgr2=0x7fb2e8102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/2156587076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 0x7fb2e8102dc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009900 tx=0x7fb2d802a410 comp rx=0 tx=0).stop 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 -- 172.21.15.129:0/2156587076 shutdown_connections 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/2156587076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2e8104550 0x7fb2e810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/2156587076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2e8103b90 0x7fb2e8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/2156587076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 0x7fb2e8102dc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 -- 172.21.15.129:0/2156587076 >> 172.21.15.129:0/2156587076 conn(0x7fb2e80fdf30 msgr2=0x7fb2e8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 -- 172.21.15.129:0/2156587076 shutdown_connections 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 -- 172.21.15.129:0/2156587076 wait complete. 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 Processor -- start 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 -- start start 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 0x7fb2e8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2e8103b90 0x7fb2e8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2e8104550 0x7fb2e81a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e810e4a0 con 0x7fb2e81029a0 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e810e320 con 0x7fb2e8104550 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.901+0000 7fb2ef70a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e810e620 con 0x7fb2e8103b90 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ed4a6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 0x7fb2e8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ed4a6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 0x7fb2e8198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59790/0 (socket says 172.21.15.129:59790) 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ed4a6640 1 -- 172.21.15.129:0/3917332517 learned_addr learned my addr 172.21.15.129:0/3917332517 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ecca5640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2e8103b90 0x7fb2e8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2edca7640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2e8104550 0x7fb2e81a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ed4a6640 1 -- 172.21.15.129:0/3917332517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2e8103b90 msgr2=0x7fb2e8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ed4a6640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2e8103b90 0x7fb2e8199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ed4a6640 1 -- 172.21.15.129:0/3917332517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2e8104550 msgr2=0x7fb2e81a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ed4a6640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2e8104550 0x7fb2e81a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:05.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ed4a6640 1 -- 172.21.15.129:0/3917332517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e81a07f0 con 0x7fb2e81029a0 2024-04-26T21:18:05.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ed4a6640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 0x7fb2e8198bd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8006010 tx=0x7fb2d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:05.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2de7fc640 1 -- 172.21.15.129:0/3917332517 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d803f030 con 0x7fb2e81029a0 2024-04-26T21:18:05.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2de7fc640 1 -- 172.21.15.129:0/3917332517 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d80318f0 con 0x7fb2e81029a0 2024-04-26T21:18:05.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e81a0a80 con 0x7fb2e81029a0 2024-04-26T21:18:05.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2de7fc640 1 -- 172.21.15.129:0/3917332517 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8035070 con 0x7fb2e81029a0 2024-04-26T21:18:05.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.902+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e81a0f70 con 0x7fb2e81029a0 2024-04-26T21:18:05.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.903+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e8109d30 con 0x7fb2e81029a0 2024-04-26T21:18:05.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.904+0000 7fb2de7fc640 1 -- 172.21.15.129:0/3917332517 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb2d8042030 con 0x7fb2e81029a0 2024-04-26T21:18:05.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.905+0000 7fb2de7fc640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2d40a3c40 0x7fb2d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:05.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.905+0000 7fb2ecca5640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2d40a3c40 0x7fb2d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:05.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.905+0000 7fb2de7fc640 1 -- 172.21.15.129:0/3917332517 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb2d80f0b00 con 0x7fb2e81029a0 2024-04-26T21:18:05.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.906+0000 7fb2ecca5640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2d40a3c40 0x7fb2d40a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb2e819a0e0 tx=0x7fb2e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:05.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:05.910+0000 7fb2de7fc640 1 -- 172.21.15.129:0/3917332517 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb2d80bd370 con 0x7fb2e81029a0 2024-04-26T21:18:06.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.089+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2e802dd20 con 0x7fb2e81029a0 2024-04-26T21:18:06.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.090+0000 7fb2de7fc640 1 -- 172.21.15.129:0/3917332517 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7fb2d80bcc70 con 0x7fb2e81029a0 2024-04-26T21:18:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:06.092 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:06.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2d40a3c40 msgr2=0x7fb2d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2d40a3c40 0x7fb2d40a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb2e819a0e0 tx=0x7fb2e400b410 comp rx=0 tx=0).stop 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 msgr2=0x7fb2e8198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 0x7fb2e8198bd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8006010 tx=0x7fb2d80042f0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 shutdown_connections 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb2d40a3c40 0x7fb2d40a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2e8104550 0x7fb2e81a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2e8103b90 0x7fb2e8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 --2- 172.21.15.129:0/3917332517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2e81029a0 0x7fb2e8198bd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 >> 172.21.15.129:0/3917332517 conn(0x7fb2e80fdf30 msgr2=0x7fb2e8106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 shutdown_connections 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.092+0000 7fb2ef70a640 1 -- 172.21.15.129:0/3917332517 wait complete. 2024-04-26T21:18:06.105 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:16:01.621500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:16:02.592849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:06.105 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:06.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 -- 172.21.15.129:0/2725490735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0102a60 msgr2=0x7f46a0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 --2- 172.21.15.129:0/2725490735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0102a60 0x7f46a0102ee0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f469c009900 tx=0x7f469c02a410 comp rx=0 tx=0).stop 2024-04-26T21:18:06.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 -- 172.21.15.129:0/2725490735 shutdown_connections 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 --2- 172.21.15.129:0/2725490735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a0103420 0x7f46a0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 --2- 172.21.15.129:0/2725490735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0102a60 0x7f46a0102ee0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 --2- 172.21.15.129:0/2725490735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46a0101870 0x7f46a0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 -- 172.21.15.129:0/2725490735 >> 172.21.15.129:0/2725490735 conn(0x7f46a00fce20 msgr2=0x7f46a00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 -- 172.21.15.129:0/2725490735 shutdown_connections 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 -- 172.21.15.129:0/2725490735 wait complete. 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 Processor -- start 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.206+0000 7f46a79bf640 1 -- start start 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a79bf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a0101870 0x7f46a006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a79bf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46a0102a60 0x7f46a019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a79bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0103420 0x7f46a019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a79bf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46a010d3a0 con 0x7f46a0103420 2024-04-26T21:18:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a79bf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46a010d220 con 0x7f46a0101870 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a79bf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46a010d520 con 0x7f46a0102a60 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a5f5c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0103420 0x7f46a019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a5f5c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0103420 0x7f46a019ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59802/0 (socket says 172.21.15.129:59802) 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a5f5c640 1 -- 172.21.15.129:0/1438847697 learned_addr learned my addr 172.21.15.129:0/1438847697 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a4f5a640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46a0102a60 0x7f46a019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a575b640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a0101870 0x7f46a006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a5f5c640 1 -- 172.21.15.129:0/1438847697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46a0102a60 msgr2=0x7f46a019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a5f5c640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46a0102a60 0x7f46a019d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a5f5c640 1 -- 172.21.15.129:0/1438847697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a0101870 msgr2=0x7f46a006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a5f5c640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a0101870 0x7f46a006bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a5f5c640 1 -- 172.21.15.129:0/1438847697 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a01a0820 con 0x7f46a0103420 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a5f5c640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0103420 0x7f46a019ff60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f469800eaa0 tx=0x7f469800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46927fc640 1 -- 172.21.15.129:0/1438847697 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469800cdb0 con 0x7f46a0103420 2024-04-26T21:18:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46927fc640 1 -- 172.21.15.129:0/1438847697 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4698005020 con 0x7f46a0103420 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a01a0b10 con 0x7f46a0103420 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46927fc640 1 -- 172.21.15.129:0/1438847697 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46980116d0 con 0x7f46a0103420 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.207+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a01a1010 con 0x7f46a0103420 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.208+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a0108c50 con 0x7f46a0103420 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.209+0000 7f46927fc640 1 -- 172.21.15.129:0/1438847697 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f46980051d0 con 0x7f46a0103420 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.210+0000 7f46927fc640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f468c0a3c40 0x7f468c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.210+0000 7f46a575b640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f468c0a3c40 0x7f468c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.210+0000 7f46927fc640 1 -- 172.21.15.129:0/1438847697 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f46980d0850 con 0x7f46a0103420 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.210+0000 7f46a575b640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f468c0a3c40 0x7f468c0a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46940058c0 tx=0x7f469400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.214+0000 7f46927fc640 1 -- 172.21.15.129:0/1438847697 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4698016030 con 0x7f46a0103420 2024-04-26T21:18:06.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.392+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46a002dcb0 con 0x7f46a0103420 2024-04-26T21:18:06.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.393+0000 7f46927fc640 1 -- 172.21.15.129:0/1438847697 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f469809d0c0 con 0x7f46a0103420 2024-04-26T21:18:06.394 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:18:06.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f468c0a3c40 msgr2=0x7f468c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f468c0a3c40 0x7f468c0a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46940058c0 tx=0x7f469400a4c0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0103420 msgr2=0x7f46a019ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0103420 0x7f46a019ff60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f469800eaa0 tx=0x7f469800ef60 comp rx=0 tx=0).stop 2024-04-26T21:18:06.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 shutdown_connections 2024-04-26T21:18:06.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f468c0a3c40 0x7f468c0a6100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46a0103420 0x7f46a019ff60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46a0102a60 0x7f46a019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 --2- 172.21.15.129:0/1438847697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46a0101870 0x7f46a006bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 >> 172.21.15.129:0/1438847697 conn(0x7f46a00fce20 msgr2=0x7f46a0105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:06.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 shutdown_connections 2024-04-26T21:18:06.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.395+0000 7f46a79bf640 1 -- 172.21.15.129:0/1438847697 wait complete. 2024-04-26T21:18:06.407 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:18:06.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.507+0000 7f24af3f0640 1 -- 172.21.15.129:0/556451325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a81033f0 msgr2=0x7f24a81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.507+0000 7f24af3f0640 1 --2- 172.21.15.129:0/556451325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a81033f0 0x7f24a81057e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009900 tx=0x7f24a402a410 comp rx=0 tx=0).stop 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.507+0000 7f24af3f0640 1 -- 172.21.15.129:0/556451325 shutdown_connections 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.507+0000 7f24af3f0640 1 --2- 172.21.15.129:0/556451325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a8108650 0x7f24a810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.507+0000 7f24af3f0640 1 --2- 172.21.15.129:0/556451325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8105d20 0x7f24a8108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.507+0000 7f24af3f0640 1 --2- 172.21.15.129:0/556451325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a81033f0 0x7f24a81057e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.507+0000 7f24af3f0640 1 -- 172.21.15.129:0/556451325 >> 172.21.15.129:0/556451325 conn(0x7f24a80fcde0 msgr2=0x7f24a80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.507+0000 7f24af3f0640 1 -- 172.21.15.129:0/556451325 shutdown_connections 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.507+0000 7f24af3f0640 1 -- 172.21.15.129:0/556451325 wait complete. 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24af3f0640 1 Processor -- start 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24af3f0640 1 -- start start 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24af3f0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a81033f0 0x7f24a8198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24af3f0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8105d20 0x7f24a81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24af3f0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a8108650 0x7f24a81a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24af3f0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24a810e520 con 0x7f24a8108650 2024-04-26T21:18:06.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24af3f0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24a810e3a0 con 0x7f24a81033f0 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24af3f0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f24a810e6a0 con 0x7f24a8105d20 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad98d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a8108650 0x7f24a81a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad98d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a8108650 0x7f24a81a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59814/0 (socket says 172.21.15.129:59814) 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad98d640 1 -- 172.21.15.129:0/4057904343 learned_addr learned my addr 172.21.15.129:0/4057904343 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad18c640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a81033f0 0x7f24a8198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ac98b640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8105d20 0x7f24a81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad98d640 1 -- 172.21.15.129:0/4057904343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8105d20 msgr2=0x7f24a81990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad98d640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8105d20 0x7f24a81990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad98d640 1 -- 172.21.15.129:0/4057904343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a81033f0 msgr2=0x7f24a8198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad98d640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a81033f0 0x7f24a8198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad98d640 1 -- 172.21.15.129:0/4057904343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a81a07d0 con 0x7f24a8108650 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.508+0000 7f24ad98d640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a8108650 0x7f24a81a0160 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f24a000cce0 tx=0x7f24a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.509+0000 7f249e7fc640 1 -- 172.21.15.129:0/4057904343 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a000f450 con 0x7f24a8108650 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.509+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a81a0ac0 con 0x7f24a8108650 2024-04-26T21:18:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.509+0000 7f249e7fc640 1 -- 172.21.15.129:0/4057904343 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a000faa0 con 0x7f24a8108650 2024-04-26T21:18:06.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.509+0000 7f249e7fc640 1 -- 172.21.15.129:0/4057904343 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0018b70 con 0x7f24a8108650 2024-04-26T21:18:06.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.509+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a81a1010 con 0x7f24a8108650 2024-04-26T21:18:06.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.510+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a80fe8d0 con 0x7f24a8108650 2024-04-26T21:18:06.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.511+0000 7f249e7fc640 1 -- 172.21.15.129:0/4057904343 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f24a0018d10 con 0x7f24a8108650 2024-04-26T21:18:06.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.511+0000 7f249e7fc640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f24940a3c40 0x7f24940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.511+0000 7f24ad18c640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f24940a3c40 0x7f24940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.511+0000 7f249e7fc640 1 -- 172.21.15.129:0/4057904343 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f24a00d0a90 con 0x7f24a8108650 2024-04-26T21:18:06.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.512+0000 7f24ad18c640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f24940a3c40 0x7f24940a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c00 tx=0x7f24a40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:06.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.516+0000 7f249e7fc640 1 -- 172.21.15.129:0/4057904343 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f24a0014030 con 0x7f24a8108650 2024-04-26T21:18:06.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.692+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f24a802dcd0 con 0x7f24a8108650 2024-04-26T21:18:06.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.692+0000 7f249e7fc640 1 -- 172.21.15.129:0/4057904343 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f24a009d300 con 0x7f24a8108650 2024-04-26T21:18:06.693 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-26T21:18:06.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f24940a3c40 msgr2=0x7f24940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f24940a3c40 0x7f24940a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c00 tx=0x7f24a40314a0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a8108650 msgr2=0x7f24a81a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a8108650 0x7f24a81a0160 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f24a000cce0 tx=0x7f24a000a620 comp rx=0 tx=0).stop 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 shutdown_connections 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f24940a3c40 0x7f24940a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a8108650 0x7f24a81a0160 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8105d20 0x7f24a81990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 --2- 172.21.15.129:0/4057904343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a81033f0 0x7f24a8198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 >> 172.21.15.129:0/4057904343 conn(0x7f24a80fcde0 msgr2=0x7f24a8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 shutdown_connections 2024-04-26T21:18:06.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.694+0000 7f24af3f0640 1 -- 172.21.15.129:0/4057904343 wait complete. 2024-04-26T21:18:06.705 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:06 smithi129 ceph-mon[29237]: pgmap v353: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:06.705 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:06 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/4076558752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:18:06.705 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:06 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1927772218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:18:06.705 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:06 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3917332517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:06.706 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:18:06.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:06 smithi171 ceph-mon[33478]: pgmap v353: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:06.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:06 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/4076558752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:18:06.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:06 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1927772218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:18:06.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:06 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3917332517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:06.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 -- 172.21.15.129:0/2036871710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f8102ac0 msgr2=0x7f47f8102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 --2- 172.21.15.129:0/2036871710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f8102ac0 0x7f47f8102f40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009900 tx=0x7f47e802a410 comp rx=0 tx=0).stop 2024-04-26T21:18:06.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 -- 172.21.15.129:0/2036871710 shutdown_connections 2024-04-26T21:18:06.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 --2- 172.21.15.129:0/2036871710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47f8103480 0x7f47f8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 --2- 172.21.15.129:0/2036871710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f8102ac0 0x7f47f8102f40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 --2- 172.21.15.129:0/2036871710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47f81018d0 0x7f47f8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 -- 172.21.15.129:0/2036871710 >> 172.21.15.129:0/2036871710 conn(0x7f47f80fce80 msgr2=0x7f47f80ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:06.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 -- 172.21.15.129:0/2036871710 shutdown_connections 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 -- 172.21.15.129:0/2036871710 wait complete. 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 Processor -- start 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 -- start start 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f81018d0 0x7f47f806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47f8102ac0 0x7f47f819d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47f8103480 0x7f47f819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47f810d370 con 0x7f47f81018d0 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f47f810d1f0 con 0x7f47f8103480 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fec93640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f47f810d4f0 con 0x7f47f8102ac0 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.806+0000 7f47fca2f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f81018d0 0x7f47f806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fca2f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f81018d0 0x7f47f806bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59820/0 (socket says 172.21.15.129:59820) 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fca2f640 1 -- 172.21.15.129:0/415082057 learned_addr learned my addr 172.21.15.129:0/415082057 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47f7fff640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47f8102ac0 0x7f47f819d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fd230640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47f8103480 0x7f47f819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fca2f640 1 -- 172.21.15.129:0/415082057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47f8102ac0 msgr2=0x7f47f819d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fca2f640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47f8102ac0 0x7f47f819d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fca2f640 1 -- 172.21.15.129:0/415082057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47f8103480 msgr2=0x7f47f819fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fca2f640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47f8103480 0x7f47f819fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fca2f640 1 -- 172.21.15.129:0/415082057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f81a07f0 con 0x7f47f81018d0 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fd230640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47f8103480 0x7f47f819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fca2f640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f81018d0 0x7f47f806bb00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47f000ccb0 tx=0x7f47f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47f5ffb640 1 -- 172.21.15.129:0/415082057 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0003b70 con 0x7f47f81018d0 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f81a0a80 con 0x7f47f81018d0 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47f5ffb640 1 -- 172.21.15.129:0/415082057 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f000ae50 con 0x7f47f81018d0 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47f5ffb640 1 -- 172.21.15.129:0/415082057 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f001d060 con 0x7f47f81018d0 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.807+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f81a1020 con 0x7f47f81018d0 2024-04-26T21:18:06.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.808+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f8108cc0 con 0x7f47f81018d0 2024-04-26T21:18:06.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.809+0000 7f47f5ffb640 1 -- 172.21.15.129:0/415082057 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f47f0004000 con 0x7f47f81018d0 2024-04-26T21:18:06.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.810+0000 7f47f5ffb640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f47e40a3c40 0x7f47e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.810+0000 7f47f7fff640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f47e40a3c40 0x7f47e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.810+0000 7f47f5ffb640 1 -- 172.21.15.129:0/415082057 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f47f00d07c0 con 0x7f47f81018d0 2024-04-26T21:18:06.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.810+0000 7f47f7fff640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f47e40a3c40 0x7f47e40a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f47e802a930 tx=0x7f47e8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:06.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.814+0000 7f47f5ffb640 1 -- 172.21.15.129:0/415082057 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f47f0014070 con 0x7f47f81018d0 2024-04-26T21:18:06.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:06 smithi150 ceph-mon[32290]: pgmap v353: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:06.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:06 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/4076558752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:18:06.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:06 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1927772218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:18:06.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:06 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3917332517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:06.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.993+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f47f802dca0 con 0x7f47f81018d0 2024-04-26T21:18:06.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.993+0000 7f47f5ffb640 1 -- 172.21.15.129:0/415082057 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f47f009d030 con 0x7f47f81018d0 2024-04-26T21:18:06.994 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-26T21:18:06.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f47e40a3c40 msgr2=0x7f47e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f47e40a3c40 0x7f47e40a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f47e802a930 tx=0x7f47e8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f81018d0 msgr2=0x7f47f806bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f81018d0 0x7f47f806bb00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47f000ccb0 tx=0x7f47f000a620 comp rx=0 tx=0).stop 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 shutdown_connections 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f47e40a3c40 0x7f47e40a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47f8103480 0x7f47f819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47f8102ac0 0x7f47f819d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 --2- 172.21.15.129:0/415082057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f81018d0 0x7f47f806bb00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 >> 172.21.15.129:0/415082057 conn(0x7f47f80fce80 msgr2=0x7f47f8105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 shutdown_connections 2024-04-26T21:18:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:06.995+0000 7f47fec93640 1 -- 172.21.15.129:0/415082057 wait complete. 2024-04-26T21:18:07.007 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-26T21:18:07.007 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.104+0000 7f0c80af8640 1 -- 172.21.15.129:0/1963858255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c104540 msgr2=0x7f0c7c10add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.104+0000 7f0c80af8640 1 --2- 172.21.15.129:0/1963858255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c104540 0x7f0c7c10add0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009900 tx=0x7f0c7402a410 comp rx=0 tx=0).stop 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 -- 172.21.15.129:0/1963858255 shutdown_connections 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 --2- 172.21.15.129:0/1963858255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c104540 0x7f0c7c10add0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 --2- 172.21.15.129:0/1963858255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c7c103b80 0x7f0c7c104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 --2- 172.21.15.129:0/1963858255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c7c102990 0x7f0c7c102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 -- 172.21.15.129:0/1963858255 >> 172.21.15.129:0/1963858255 conn(0x7f0c7c0fdee0 msgr2=0x7f0c7c100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 -- 172.21.15.129:0/1963858255 shutdown_connections 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 -- 172.21.15.129:0/1963858255 wait complete. 2024-04-26T21:18:07.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 Processor -- start 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 -- start start 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c7c102990 0x7f0c7c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c103b80 0x7f0c7c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.105+0000 7f0c80af8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c7c104540 0x7f0c7c1a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c80af8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c10e4a0 con 0x7f0c7c103b80 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c80af8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c10e320 con 0x7f0c7c102990 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c80af8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c10e620 con 0x7f0c7c104540 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c103b80 0x7f0c7c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c103b80 0x7f0c7c199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59840/0 (socket says 172.21.15.129:59840) 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c79d9b640 1 -- 172.21.15.129:0/2583406038 learned_addr learned my addr 172.21.15.129:0/2583406038 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c7a59c640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c7c102990 0x7f0c7c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c79d9b640 1 -- 172.21.15.129:0/2583406038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c7c104540 msgr2=0x7f0c7c1a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c7ad9d640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c7c104540 0x7f0c7c1a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c79d9b640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c7c104540 0x7f0c7c1a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c79d9b640 1 -- 172.21.15.129:0/2583406038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c7c102990 msgr2=0x7f0c7c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c79d9b640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c7c102990 0x7f0c7c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c79d9b640 1 -- 172.21.15.129:0/2583406038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7c1a0760 con 0x7f0c7c103b80 2024-04-26T21:18:07.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c79d9b640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c103b80 0x7f0c7c199080 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000d950 tx=0x7f0c7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:07.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c6b7fe640 1 -- 172.21.15.129:0/2583406038 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70017030 con 0x7f0c7c103b80 2024-04-26T21:18:07.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7c1a0a50 con 0x7f0c7c103b80 2024-04-26T21:18:07.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c6b7fe640 1 -- 172.21.15.129:0/2583406038 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c70004020 con 0x7f0c7c103b80 2024-04-26T21:18:07.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c6b7fe640 1 -- 172.21.15.129:0/2583406038 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c700047a0 con 0x7f0c7c103b80 2024-04-26T21:18:07.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.106+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c7c1a0ff0 con 0x7f0c7c103b80 2024-04-26T21:18:07.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.107+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c7c069750 con 0x7f0c7c103b80 2024-04-26T21:18:07.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.108+0000 7f0c6b7fe640 1 -- 172.21.15.129:0/2583406038 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0c70009a30 con 0x7f0c7c103b80 2024-04-26T21:18:07.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.109+0000 7f0c6b7fe640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3c40 0x7f0c640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.109+0000 7f0c7a59c640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3c40 0x7f0c640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.109+0000 7f0c6b7fe640 1 -- 172.21.15.129:0/2583406038 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0c700d0630 con 0x7f0c7c103b80 2024-04-26T21:18:07.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.109+0000 7f0c7a59c640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3c40 0x7f0c640a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c1039e0 tx=0x7f0c6c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:07.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.113+0000 7f0c6b7fe640 1 -- 172.21.15.129:0/2583406038 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0c7009cea0 con 0x7f0c7c103b80 2024-04-26T21:18:07.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.328+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c7c109cb0 con 0x7f0c7c103b80 2024-04-26T21:18:07.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.328+0000 7f0c6b7fe640 1 -- 172.21.15.129:0/2583406038 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f0c7009c7a0 con 0x7f0c7c103b80 2024-04-26T21:18:07.329 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:07.330 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:18:07.330 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3c40 msgr2=0x7f0c640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3c40 0x7f0c640a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c1039e0 tx=0x7f0c6c009500 comp rx=0 tx=0).stop 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c103b80 msgr2=0x7f0c7c199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c103b80 0x7f0c7c199080 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000d950 tx=0x7f0c7000de10 comp rx=0 tx=0).stop 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 shutdown_connections 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0c640a3c40 0x7f0c640a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c7c104540 0x7f0c7c1a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c7c103b80 0x7f0c7c199080 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 --2- 172.21.15.129:0/2583406038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c7c102990 0x7f0c7c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 >> 172.21.15.129:0/2583406038 conn(0x7f0c7c0fdee0 msgr2=0x7f0c7c106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 shutdown_connections 2024-04-26T21:18:07.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.331+0000 7f0c80af8640 1 -- 172.21.15.129:0/2583406038 wait complete. 2024-04-26T21:18:07.343 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-26T21:18:07.343 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-26T21:18:07.343 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:18:07.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 -- 172.21.15.129:0/2765086831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 msgr2=0x7f5f78101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/2765086831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 0x7f5f78101430 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009900 tx=0x7f5f6802a410 comp rx=0 tx=0).stop 2024-04-26T21:18:07.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 -- 172.21.15.129:0/2765086831 shutdown_connections 2024-04-26T21:18:07.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/2765086831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78102200 0x7f5f78102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/2765086831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f78102bc0 0x7f5f78109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/2765086831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 0x7f5f78101430 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 -- 172.21.15.129:0/2765086831 >> 172.21.15.129:0/2765086831 conn(0x7f5f780fc5a0 msgr2=0x7f5f780fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 -- 172.21.15.129:0/2765086831 shutdown_connections 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 -- 172.21.15.129:0/2765086831 wait complete. 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 Processor -- start 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.442+0000 7f5f7dd14640 1 -- start start 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f7dd14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 0x7f5f7819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f7dd14640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f78102200 0x7f5f7819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f7dd14640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78102bc0 0x7f5f781a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f7dd14640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7810e510 con 0x7f5f78101010 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f7dd14640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7810e390 con 0x7f5f78102bc0 2024-04-26T21:18:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f7dd14640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7810e690 con 0x7f5f78102200 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 0x7f5f7819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 0x7f5f7819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59868/0 (socket says 172.21.15.129:59868) 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f777fe640 1 -- 172.21.15.129:0/1780193638 learned_addr learned my addr 172.21.15.129:0/1780193638 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f77fff640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78102bc0 0x7f5f781a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f76ffd640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f78102200 0x7f5f7819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f777fe640 1 -- 172.21.15.129:0/1780193638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f78102200 msgr2=0x7f5f7819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f777fe640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f78102200 0x7f5f7819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f777fe640 1 -- 172.21.15.129:0/1780193638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78102bc0 msgr2=0x7f5f781a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f777fe640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78102bc0 0x7f5f781a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f777fe640 1 -- 172.21.15.129:0/1780193638 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f781a2a20 con 0x7f5f78101010 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f777fe640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 0x7f5f7819ae50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5f68006010 tx=0x7f5f680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f74ff9640 1 -- 172.21.15.129:0/1780193638 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6803f030 con 0x7f5f78101010 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f74ff9640 1 -- 172.21.15.129:0/1780193638 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f68003cb0 con 0x7f5f78101010 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f781a2cb0 con 0x7f5f78101010 2024-04-26T21:18:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f74ff9640 1 -- 172.21.15.129:0/1780193638 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68035070 con 0x7f5f78101010 2024-04-26T21:18:07.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.443+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f781a31a0 con 0x7f5f78101010 2024-04-26T21:18:07.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.444+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f781a3370 con 0x7f5f78101010 2024-04-26T21:18:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.445+0000 7f5f74ff9640 1 -- 172.21.15.129:0/1780193638 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5f68042030 con 0x7f5f78101010 2024-04-26T21:18:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.446+0000 7f5f74ff9640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f600a3c40 0x7f5f600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.446+0000 7f5f76ffd640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f600a3c40 0x7f5f600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.446+0000 7f5f74ff9640 1 -- 172.21.15.129:0/1780193638 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5f680f0be0 con 0x7f5f78101010 2024-04-26T21:18:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.447+0000 7f5f76ffd640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f600a3c40 0x7f5f600a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c005fd0 tx=0x7f5f6c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:07.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.450+0000 7f5f74ff9640 1 -- 172.21.15.129:0/1780193638 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5f680bd450 con 0x7f5f78101010 2024-04-26T21:18:07.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.625+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f7810c340 con 0x7f5f600a3c40 2024-04-26T21:18:07.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.632+0000 7f5f74ff9640 1 -- 172.21.15.129:0/1780193638 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7f5f7810c340 con 0x7f5f600a3c40 2024-04-26T21:18:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:07.637 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-26T21:18:07.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f600a3c40 msgr2=0x7f5f600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f600a3c40 0x7f5f600a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c005fd0 tx=0x7f5f6c005850 comp rx=0 tx=0).stop 2024-04-26T21:18:07.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 msgr2=0x7f5f7819ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 0x7f5f7819ae50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5f68006010 tx=0x7f5f680042f0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 shutdown_connections 2024-04-26T21:18:07.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5f600a3c40 0x7f5f600a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78102bc0 0x7f5f781a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f78102200 0x7f5f7819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 --2- 172.21.15.129:0/1780193638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f78101010 0x7f5f7819ae50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 >> 172.21.15.129:0/1780193638 conn(0x7f5f780fc5a0 msgr2=0x7f5f7810ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:07.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 shutdown_connections 2024-04-26T21:18:07.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.635+0000 7f5f7dd14640 1 -- 172.21.15.129:0/1780193638 wait complete. 2024-04-26T21:18:07.646 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:07 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1438847697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:18:07.646 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:07 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/4057904343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:18:07.646 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:07 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/415082057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:18:07.646 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:07 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2583406038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:18:07.647 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-04-26T21:18:07.308293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643932,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":369,"num_read_kb":284,"num_write":260,"num_write_kb":3015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360696,"kb_used_data":11484,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581128,"statfs":{"total":1151940427776,"available":1151571075072,"internally_reserved":0,"allocated":11759616,"data_stored":7437620,"data_compressed":182064,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002752"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359578+0000","last_change":"2024-04-26T21:16:03.700581+0000","last_active":"2024-04-26T21:16:56.359578+0000","last_peered":"2024-04-26T21:16:56.359578+0000","last_clean":"2024-04-26T21:16:56.359578+0000","last_became_active":"2024-04-26T21:16:03.700280+0000","last_became_peered":"2024-04-26T21:16:03.700280+0000","last_unstale":"2024-04-26T21:16:56.359578+0000","last_undegraded":"2024-04-26T21:16:56.359578+0000","last_fullsized":"2024-04-26T21:16:56.359578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:01.329784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368025+0000","last_change":"2024-04-26T21:16:03.702058+0000","last_active":"2024-04-26T21:16:56.368025+0000","last_peered":"2024-04-26T21:16:56.368025+0000","last_clean":"2024-04-26T21:16:56.368025+0000","last_became_active":"2024-04-26T21:16:03.701728+0000","last_became_peered":"2024-04-26T21:16:03.701728+0000","last_unstale":"2024-04-26T21:16:56.368025+0000","last_undegraded":"2024-04-26T21:16:56.368025+0000","last_fullsized":"2024-04-26T21:16:56.368025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:12.718225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367712+0000","last_change":"2024-04-26T21:16:03.710461+0000","last_active":"2024-04-26T21:16:56.367712+0000","last_peered":"2024-04-26T21:16:56.367712+0000","last_clean":"2024-04-26T21:16:56.367712+0000","last_became_active":"2024-04-26T21:16:03.710130+0000","last_became_peered":"2024-04-26T21:16:03.710130+0000","last_unstale":"2024-04-26T21:16:56.367712+0000","last_undegraded":"2024-04-26T21:16:56.367712+0000","last_fullsized":"2024-04-26T21:16:56.367712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:21.294677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360217+0000","last_change":"2024-04-26T21:16:03.706836+0000","last_active":"2024-04-26T21:16:56.360217+0000","last_peered":"2024-04-26T21:16:56.360217+0000","last_clean":"2024-04-26T21:16:56.360217+0000","last_became_active":"2024-04-26T21:16:03.706515+0000","last_became_peered":"2024-04-26T21:16:03.706515+0000","last_unstale":"2024-04-26T21:16:56.360217+0000","last_undegraded":"2024-04-26T21:16:56.360217+0000","last_fullsized":"2024-04-26T21:16:56.360217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:56.198337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362204+0000","last_change":"2024-04-26T21:16:03.704865+0000","last_active":"2024-04-26T21:16:56.362204+0000","last_peered":"2024-04-26T21:16:56.362204+0000","last_clean":"2024-04-26T21:16:56.362204+0000","last_became_active":"2024-04-26T21:16:03.704560+0000","last_became_peered":"2024-04-26T21:16:03.704560+0000","last_unstale":"2024-04-26T21:16:56.362204+0000","last_undegraded":"2024-04-26T21:16:56.362204+0000","last_fullsized":"2024-04-26T21:16:56.362204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:51:03.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712989+0000","last_change":"2024-04-26T21:16:03.700265+0000","last_active":"2024-04-26T21:16:56.712989+0000","last_peered":"2024-04-26T21:16:56.712989+0000","last_clean":"2024-04-26T21:16:56.712989+0000","last_became_active":"2024-04-26T21:16:03.699878+0000","last_became_peered":"2024-04-26T21:16:03.699878+0000","last_unstale":"2024-04-26T21:16:56.712989+0000","last_undegraded":"2024-04-26T21:16:56.712989+0000","last_fullsized":"2024-04-26T21:16:56.712989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:12.907254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361631+0000","last_change":"2024-04-26T21:16:03.697569+0000","last_active":"2024-04-26T21:16:56.361631+0000","last_peered":"2024-04-26T21:16:56.361631+0000","last_clean":"2024-04-26T21:16:56.361631+0000","last_became_active":"2024-04-26T21:16:03.697262+0000","last_became_peered":"2024-04-26T21:16:03.697262+0000","last_unstale":"2024-04-26T21:16:56.361631+0000","last_undegraded":"2024-04-26T21:16:56.361631+0000","last_fullsized":"2024-04-26T21:16:56.361631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:28.645750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363218+0000","last_change":"2024-04-26T21:16:03.705788+0000","last_active":"2024-04-26T21:16:56.363218+0000","last_peered":"2024-04-26T21:16:56.363218+0000","last_clean":"2024-04-26T21:16:56.363218+0000","last_became_active":"2024-04-26T21:16:03.705479+0000","last_became_peered":"2024-04-26T21:16:03.705479+0000","last_unstale":"2024-04-26T21:16:56.363218+0000","last_undegraded":"2024-04-26T21:16:56.363218+0000","last_fullsized":"2024-04-26T21:16:56.363218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:27.306777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713655+0000","last_change":"2024-04-26T21:16:03.709716+0000","last_active":"2024-04-26T21:16:56.713655+0000","last_peered":"2024-04-26T21:16:56.713655+0000","last_clean":"2024-04-26T21:16:56.713655+0000","last_became_active":"2024-04-26T21:16:03.709219+0000","last_became_peered":"2024-04-26T21:16:03.709219+0000","last_unstale":"2024-04-26T21:16:56.713655+0000","last_undegraded":"2024-04-26T21:16:56.713655+0000","last_fullsized":"2024-04-26T21:16:56.713655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:18.936075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713179+0000","last_change":"2024-04-26T21:16:03.694745+0000","last_active":"2024-04-26T21:16:56.713179+0000","last_peered":"2024-04-26T21:16:56.713179+0000","last_clean":"2024-04-26T21:16:56.713179+0000","last_became_active":"2024-04-26T21:16:03.694044+0000","last_became_peered":"2024-04-26T21:16:03.694044+0000","last_unstale":"2024-04-26T21:16:56.713179+0000","last_undegraded":"2024-04-26T21:16:56.713179+0000","last_fullsized":"2024-04-26T21:16:56.713179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:13.060467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362033+0000","last_change":"2024-04-26T21:16:03.706434+0000","last_active":"2024-04-26T21:16:56.362033+0000","last_peered":"2024-04-26T21:16:56.362033+0000","last_clean":"2024-04-26T21:16:56.362033+0000","last_became_active":"2024-04-26T21:16:03.706093+0000","last_became_peered":"2024-04-26T21:16:03.706093+0000","last_unstale":"2024-04-26T21:16:56.362033+0000","last_undegraded":"2024-04-26T21:16:56.362033+0000","last_fullsized":"2024-04-26T21:16:56.362033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:04.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360084+0000","last_change":"2024-04-26T21:16:03.707672+0000","last_active":"2024-04-26T21:16:56.360084+0000","last_peered":"2024-04-26T21:16:56.360084+0000","last_clean":"2024-04-26T21:16:56.360084+0000","last_became_active":"2024-04-26T21:16:03.707344+0000","last_became_peered":"2024-04-26T21:16:03.707344+0000","last_unstale":"2024-04-26T21:16:56.360084+0000","last_undegraded":"2024-04-26T21:16:56.360084+0000","last_fullsized":"2024-04-26T21:16:56.360084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:07.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361281+0000","last_change":"2024-04-26T21:16:03.699816+0000","last_active":"2024-04-26T21:16:56.361281+0000","last_peered":"2024-04-26T21:16:56.361281+0000","last_clean":"2024-04-26T21:16:56.361281+0000","last_became_active":"2024-04-26T21:16:03.699514+0000","last_became_peered":"2024-04-26T21:16:03.699514+0000","last_unstale":"2024-04-26T21:16:56.361281+0000","last_undegraded":"2024-04-26T21:16:56.361281+0000","last_fullsized":"2024-04-26T21:16:56.361281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:59.230438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367152+0000","last_change":"2024-04-26T21:16:03.709627+0000","last_active":"2024-04-26T21:16:56.367152+0000","last_peered":"2024-04-26T21:16:56.367152+0000","last_clean":"2024-04-26T21:16:56.367152+0000","last_became_active":"2024-04-26T21:16:03.709238+0000","last_became_peered":"2024-04-26T21:16:03.709238+0000","last_unstale":"2024-04-26T21:16:56.367152+0000","last_undegraded":"2024-04-26T21:16:56.367152+0000","last_fullsized":"2024-04-26T21:16:56.367152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:55.303059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.366111+0000","last_change":"2024-04-26T21:16:03.702275+0000","last_active":"2024-04-26T21:16:56.366111+0000","last_peered":"2024-04-26T21:16:56.366111+0000","last_clean":"2024-04-26T21:16:56.366111+0000","last_became_active":"2024-04-26T21:16:03.701966+0000","last_became_peered":"2024-04-26T21:16:03.701966+0000","last_unstale":"2024-04-26T21:16:56.366111+0000","last_undegraded":"2024-04-26T21:16:56.366111+0000","last_fullsized":"2024-04-26T21:16:56.366111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:02.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713281+0000","last_change":"2024-04-26T21:16:03.711979+0000","last_active":"2024-04-26T21:16:56.713281+0000","last_peered":"2024-04-26T21:16:56.713281+0000","last_clean":"2024-04-26T21:16:56.713281+0000","last_became_active":"2024-04-26T21:16:03.711637+0000","last_became_peered":"2024-04-26T21:16:03.711637+0000","last_unstale":"2024-04-26T21:16:56.713281+0000","last_undegraded":"2024-04-26T21:16:56.713281+0000","last_fullsized":"2024-04-26T21:16:56.713281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:27.817993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359639+0000","last_change":"2024-04-26T21:16:03.701303+0000","last_active":"2024-04-26T21:16:56.359639+0000","last_peered":"2024-04-26T21:16:56.359639+0000","last_clean":"2024-04-26T21:16:56.359639+0000","last_became_active":"2024-04-26T21:16:03.701001+0000","last_became_peered":"2024-04-26T21:16:03.701001+0000","last_unstale":"2024-04-26T21:16:56.359639+0000","last_undegraded":"2024-04-26T21:16:56.359639+0000","last_fullsized":"2024-04-26T21:16:56.359639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:08.349915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362455+0000","last_change":"2024-04-26T21:16:03.696775+0000","last_active":"2024-04-26T21:16:56.362455+0000","last_peered":"2024-04-26T21:16:56.362455+0000","last_clean":"2024-04-26T21:16:56.362455+0000","last_became_active":"2024-04-26T21:16:03.696433+0000","last_became_peered":"2024-04-26T21:16:03.696433+0000","last_unstale":"2024-04-26T21:16:56.362455+0000","last_undegraded":"2024-04-26T21:16:56.362455+0000","last_fullsized":"2024-04-26T21:16:56.362455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:50.029169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361906+0000","last_change":"2024-04-26T21:16:03.698188+0000","last_active":"2024-04-26T21:16:56.361906+0000","last_peered":"2024-04-26T21:16:56.361906+0000","last_clean":"2024-04-26T21:16:56.361906+0000","last_became_active":"2024-04-26T21:16:03.697882+0000","last_became_peered":"2024-04-26T21:16:03.697882+0000","last_unstale":"2024-04-26T21:16:56.361906+0000","last_undegraded":"2024-04-26T21:16:56.361906+0000","last_fullsized":"2024-04-26T21:16:56.361906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:04.737619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362709+0000","last_change":"2024-04-26T21:16:03.275296+0000","last_active":"2024-04-26T21:16:56.362709+0000","last_peered":"2024-04-26T21:16:56.362709+0000","last_clean":"2024-04-26T21:16:56.362709+0000","last_became_active":"2024-04-26T21:16:03.274733+0000","last_became_peered":"2024-04-26T21:16:03.274733+0000","last_unstale":"2024-04-26T21:16:56.362709+0000","last_undegraded":"2024-04-26T21:16:56.362709+0000","last_fullsized":"2024-04-26T21:16:56.362709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:16.512934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363299+0000","last_change":"2024-04-26T21:16:03.705823+0000","last_active":"2024-04-26T21:16:56.363299+0000","last_peered":"2024-04-26T21:16:56.363299+0000","last_clean":"2024-04-26T21:16:56.363299+0000","last_became_active":"2024-04-26T21:16:03.705512+0000","last_became_peered":"2024-04-26T21:16:03.705512+0000","last_unstale":"2024-04-26T21:16:56.363299+0000","last_undegraded":"2024-04-26T21:16:56.363299+0000","last_fullsized":"2024-04-26T21:16:56.363299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:17.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364347+0000","last_change":"2024-04-26T21:16:03.285008+0000","last_active":"2024-04-26T21:16:56.364347+0000","last_peered":"2024-04-26T21:16:56.364347+0000","last_clean":"2024-04-26T21:16:56.364347+0000","last_became_active":"2024-04-26T21:16:03.284679+0000","last_became_peered":"2024-04-26T21:16:03.284679+0000","last_unstale":"2024-04-26T21:16:56.364347+0000","last_undegraded":"2024-04-26T21:16:56.364347+0000","last_fullsized":"2024-04-26T21:16:56.364347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:35.404086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368858+0000","last_change":"2024-04-26T21:16:03.710571+0000","last_active":"2024-04-26T21:16:56.368858+0000","last_peered":"2024-04-26T21:16:56.368858+0000","last_clean":"2024-04-26T21:16:56.368858+0000","last_became_active":"2024-04-26T21:16:03.710237+0000","last_became_peered":"2024-04-26T21:16:03.710237+0000","last_unstale":"2024-04-26T21:16:56.368858+0000","last_undegraded":"2024-04-26T21:16:56.368858+0000","last_fullsized":"2024-04-26T21:16:56.368858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:53.188554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369209+0000","last_change":"2024-04-26T21:16:56.368746+0000","last_active":"2024-04-26T21:16:56.369209+0000","last_peered":"2024-04-26T21:16:56.369209+0000","last_clean":"2024-04-26T21:16:56.369209+0000","last_became_active":"2024-04-26T21:16:56.365062+0000","last_became_peered":"2024-04-26T21:16:56.365062+0000","last_unstale":"2024-04-26T21:16:56.369209+0000","last_undegraded":"2024-04-26T21:16:56.369209+0000","last_fullsized":"2024-04-26T21:16:56.369209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:36.165146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713534+0000","last_change":"2024-04-26T21:16:03.711355+0000","last_active":"2024-04-26T21:16:56.713534+0000","last_peered":"2024-04-26T21:16:56.713534+0000","last_clean":"2024-04-26T21:16:56.713534+0000","last_became_active":"2024-04-26T21:16:03.710885+0000","last_became_peered":"2024-04-26T21:16:03.710885+0000","last_unstale":"2024-04-26T21:16:56.713534+0000","last_undegraded":"2024-04-26T21:16:56.713534+0000","last_fullsized":"2024-04-26T21:16:56.713534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:10.515590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362768+0000","last_change":"2024-04-26T21:16:03.278484+0000","last_active":"2024-04-26T21:16:56.362768+0000","last_peered":"2024-04-26T21:16:56.362768+0000","last_clean":"2024-04-26T21:16:56.362768+0000","last_became_active":"2024-04-26T21:16:03.277402+0000","last_became_peered":"2024-04-26T21:16:03.277402+0000","last_unstale":"2024-04-26T21:16:56.362768+0000","last_undegraded":"2024-04-26T21:16:56.362768+0000","last_fullsized":"2024-04-26T21:16:56.362768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:21.807942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714671+0000","last_change":"2024-04-26T21:16:56.714671+0000","last_active":"2024-04-26T21:16:56.714671+0000","last_peered":"2024-04-26T21:16:56.714671+0000","last_clean":"2024-04-26T21:16:56.714671+0000","last_became_active":"2024-04-26T21:16:56.714209+0000","last_became_peered":"2024-04-26T21:16:56.714209+0000","last_unstale":"2024-04-26T21:16:56.714671+0000","last_undegraded":"2024-04-26T21:16:56.714671+0000","last_fullsized":"2024-04-26T21:16:56.714671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:44.790915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363442+0000","last_change":"2024-04-26T21:16:03.277219+0000","last_active":"2024-04-26T21:16:56.363442+0000","last_peered":"2024-04-26T21:16:56.363442+0000","last_clean":"2024-04-26T21:16:56.363442+0000","last_became_active":"2024-04-26T21:16:03.276885+0000","last_became_peered":"2024-04-26T21:16:03.276885+0000","last_unstale":"2024-04-26T21:16:56.363442+0000","last_undegraded":"2024-04-26T21:16:56.363442+0000","last_fullsized":"2024-04-26T21:16:56.363442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:45.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362963+0000","last_change":"2024-04-26T21:16:03.703453+0000","last_active":"2024-04-26T21:16:56.362963+0000","last_peered":"2024-04-26T21:16:56.362963+0000","last_clean":"2024-04-26T21:16:56.362963+0000","last_became_active":"2024-04-26T21:16:03.703161+0000","last_became_peered":"2024-04-26T21:16:03.703161+0000","last_unstale":"2024-04-26T21:16:56.362963+0000","last_undegraded":"2024-04-26T21:16:56.362963+0000","last_fullsized":"2024-04-26T21:16:56.362963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.999243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712837+0000","last_change":"2024-04-26T21:16:03.274815+0000","last_active":"2024-04-26T21:16:56.712837+0000","last_peered":"2024-04-26T21:16:56.712837+0000","last_clean":"2024-04-26T21:16:56.712837+0000","last_became_active":"2024-04-26T21:16:03.274509+0000","last_became_peered":"2024-04-26T21:16:03.274509+0000","last_unstale":"2024-04-26T21:16:56.712837+0000","last_undegraded":"2024-04-26T21:16:56.712837+0000","last_fullsized":"2024-04-26T21:16:56.712837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:27.212714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.371100+0000","last_change":"2024-04-26T21:16:56.371100+0000","last_active":"2024-04-26T21:16:56.371100+0000","last_peered":"2024-04-26T21:16:56.371100+0000","last_clean":"2024-04-26T21:16:56.371100+0000","last_became_active":"2024-04-26T21:16:56.370711+0000","last_became_peered":"2024-04-26T21:16:56.370711+0000","last_unstale":"2024-04-26T21:16:56.371100+0000","last_undegraded":"2024-04-26T21:16:56.371100+0000","last_fullsized":"2024-04-26T21:16:56.371100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:00.143815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360698+0000","last_change":"2024-04-26T21:16:03.270417+0000","last_active":"2024-04-26T21:16:56.360698+0000","last_peered":"2024-04-26T21:16:56.360698+0000","last_clean":"2024-04-26T21:16:56.360698+0000","last_became_active":"2024-04-26T21:16:03.270066+0000","last_became_peered":"2024-04-26T21:16:03.270066+0000","last_unstale":"2024-04-26T21:16:56.360698+0000","last_undegraded":"2024-04-26T21:16:56.360698+0000","last_fullsized":"2024-04-26T21:16:56.360698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:46.794698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363117+0000","last_change":"2024-04-26T21:16:03.700221+0000","last_active":"2024-04-26T21:16:56.363117+0000","last_peered":"2024-04-26T21:16:56.363117+0000","last_clean":"2024-04-26T21:16:56.363117+0000","last_became_active":"2024-04-26T21:16:03.699798+0000","last_became_peered":"2024-04-26T21:16:03.699798+0000","last_unstale":"2024-04-26T21:16:56.363117+0000","last_undegraded":"2024-04-26T21:16:56.363117+0000","last_fullsized":"2024-04-26T21:16:56.363117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:52.083700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362074+0000","last_change":"2024-04-26T21:16:03.282355+0000","last_active":"2024-04-26T21:16:56.362074+0000","last_peered":"2024-04-26T21:16:56.362074+0000","last_clean":"2024-04-26T21:16:56.362074+0000","last_became_active":"2024-04-26T21:16:03.281169+0000","last_became_peered":"2024-04-26T21:16:03.281169+0000","last_unstale":"2024-04-26T21:16:56.362074+0000","last_undegraded":"2024-04-26T21:16:56.362074+0000","last_fullsized":"2024-04-26T21:16:56.362074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:55.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714327+0000","last_change":"2024-04-26T21:16:56.714327+0000","last_active":"2024-04-26T21:16:56.714327+0000","last_peered":"2024-04-26T21:16:56.714327+0000","last_clean":"2024-04-26T21:16:56.714327+0000","last_became_active":"2024-04-26T21:16:56.713967+0000","last_became_peered":"2024-04-26T21:16:56.713967+0000","last_unstale":"2024-04-26T21:16:56.714327+0000","last_undegraded":"2024-04-26T21:16:56.714327+0000","last_fullsized":"2024-04-26T21:16:56.714327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:32.475044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713294+0000","last_change":"2024-04-26T21:16:03.284948+0000","last_active":"2024-04-26T21:16:56.713294+0000","last_peered":"2024-04-26T21:16:56.713294+0000","last_clean":"2024-04-26T21:16:56.713294+0000","last_became_active":"2024-04-26T21:16:03.284570+0000","last_became_peered":"2024-04-26T21:16:03.284570+0000","last_unstale":"2024-04-26T21:16:56.713294+0000","last_undegraded":"2024-04-26T21:16:56.713294+0000","last_fullsized":"2024-04-26T21:16:56.713294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:22.883834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713580+0000","last_change":"2024-04-26T21:16:03.710359+0000","last_active":"2024-04-26T21:16:56.713580+0000","last_peered":"2024-04-26T21:16:56.713580+0000","last_clean":"2024-04-26T21:16:56.713580+0000","last_became_active":"2024-04-26T21:16:03.710053+0000","last_became_peered":"2024-04-26T21:16:03.710053+0000","last_unstale":"2024-04-26T21:16:56.713580+0000","last_undegraded":"2024-04-26T21:16:56.713580+0000","last_fullsized":"2024-04-26T21:16:56.713580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:45.981083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361874+0000","last_change":"2024-04-26T21:16:03.278787+0000","last_active":"2024-04-26T21:16:56.361874+0000","last_peered":"2024-04-26T21:16:56.361874+0000","last_clean":"2024-04-26T21:16:56.361874+0000","last_became_active":"2024-04-26T21:16:03.278483+0000","last_became_peered":"2024-04-26T21:16:03.278483+0000","last_unstale":"2024-04-26T21:16:56.361874+0000","last_undegraded":"2024-04-26T21:16:56.361874+0000","last_fullsized":"2024-04-26T21:16:56.361874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:40.303372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372769+0000","last_change":"2024-04-26T21:16:56.372769+0000","last_active":"2024-04-26T21:16:56.372769+0000","last_peered":"2024-04-26T21:16:56.372769+0000","last_clean":"2024-04-26T21:16:56.372769+0000","last_became_active":"2024-04-26T21:16:56.372393+0000","last_became_peered":"2024-04-26T21:16:56.372393+0000","last_unstale":"2024-04-26T21:16:56.372769+0000","last_undegraded":"2024-04-26T21:16:56.372769+0000","last_fullsized":"2024-04-26T21:16:56.372769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:29.818635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364174+0000","last_change":"2024-04-26T21:16:03.277387+0000","last_active":"2024-04-26T21:16:56.364174+0000","last_peered":"2024-04-26T21:16:56.364174+0000","last_clean":"2024-04-26T21:16:56.364174+0000","last_became_active":"2024-04-26T21:16:03.274765+0000","last_became_peered":"2024-04-26T21:16:03.274765+0000","last_unstale":"2024-04-26T21:16:56.364174+0000","last_undegraded":"2024-04-26T21:16:56.364174+0000","last_fullsized":"2024-04-26T21:16:56.364174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:51.081883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713473+0000","last_change":"2024-04-26T21:16:03.710977+0000","last_active":"2024-04-26T21:16:56.713473+0000","last_peered":"2024-04-26T21:16:56.713473+0000","last_clean":"2024-04-26T21:16:56.713473+0000","last_became_active":"2024-04-26T21:16:03.710284+0000","last_became_peered":"2024-04-26T21:16:03.710284+0000","last_unstale":"2024-04-26T21:16:56.713473+0000","last_undegraded":"2024-04-26T21:16:56.713473+0000","last_fullsized":"2024-04-26T21:16:56.713473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:46.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'25","reported_seq":281,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:18:01.869515+0000","last_change":"2024-04-26T21:16:03.276865+0000","last_active":"2024-04-26T21:18:01.869515+0000","last_peered":"2024-04-26T21:18:01.869515+0000","last_clean":"2024-04-26T21:18:01.869515+0000","last_became_active":"2024-04-26T21:16:03.276564+0000","last_became_peered":"2024-04-26T21:16:03.276564+0000","last_unstale":"2024-04-26T21:18:01.869515+0000","last_undegraded":"2024-04-26T21:18:01.869515+0000","last_fullsized":"2024-04-26T21:18:01.869515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:33.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52532,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":237,"num_read_kb":169,"num_write":25,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362088+0000","last_change":"2024-04-26T21:16:03.700441+0000","last_active":"2024-04-26T21:16:56.362088+0000","last_peered":"2024-04-26T21:16:56.362088+0000","last_clean":"2024-04-26T21:16:56.362088+0000","last_became_active":"2024-04-26T21:16:03.700142+0000","last_became_peered":"2024-04-26T21:16:03.700142+0000","last_unstale":"2024-04-26T21:16:56.362088+0000","last_undegraded":"2024-04-26T21:16:56.362088+0000","last_fullsized":"2024-04-26T21:16:56.362088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:35.166603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361872+0000","last_change":"2024-04-26T21:16:03.280144+0000","last_active":"2024-04-26T21:16:56.361872+0000","last_peered":"2024-04-26T21:16:56.361872+0000","last_clean":"2024-04-26T21:16:56.361872+0000","last_became_active":"2024-04-26T21:16:03.279844+0000","last_became_peered":"2024-04-26T21:16:03.279844+0000","last_unstale":"2024-04-26T21:16:56.361872+0000","last_undegraded":"2024-04-26T21:16:56.361872+0000","last_fullsized":"2024-04-26T21:16:56.361872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:00.704181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361493+0000","last_change":"2024-04-26T21:16:03.701732+0000","last_active":"2024-04-26T21:16:56.361493+0000","last_peered":"2024-04-26T21:16:56.361493+0000","last_clean":"2024-04-26T21:16:56.361493+0000","last_became_active":"2024-04-26T21:16:03.701424+0000","last_became_peered":"2024-04-26T21:16:03.701424+0000","last_unstale":"2024-04-26T21:16:56.361493+0000","last_undegraded":"2024-04-26T21:16:56.361493+0000","last_fullsized":"2024-04-26T21:16:56.361493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:02.293086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363077+0000","last_change":"2024-04-26T21:16:03.276816+0000","last_active":"2024-04-26T21:16:56.363077+0000","last_peered":"2024-04-26T21:16:56.363077+0000","last_clean":"2024-04-26T21:16:56.363077+0000","last_became_active":"2024-04-26T21:16:03.276471+0000","last_became_peered":"2024-04-26T21:16:03.276471+0000","last_unstale":"2024-04-26T21:16:56.363077+0000","last_undegraded":"2024-04-26T21:16:56.363077+0000","last_fullsized":"2024-04-26T21:16:56.363077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:44.630638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361866+0000","last_change":"2024-04-26T21:16:03.701002+0000","last_active":"2024-04-26T21:16:56.361866+0000","last_peered":"2024-04-26T21:16:56.361866+0000","last_clean":"2024-04-26T21:16:56.361866+0000","last_became_active":"2024-04-26T21:16:03.700649+0000","last_became_peered":"2024-04-26T21:16:03.700649+0000","last_unstale":"2024-04-26T21:16:56.361866+0000","last_undegraded":"2024-04-26T21:16:56.361866+0000","last_fullsized":"2024-04-26T21:16:56.361866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:45.571735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359679+0000","last_change":"2024-04-26T21:16:03.273727+0000","last_active":"2024-04-26T21:16:56.359679+0000","last_peered":"2024-04-26T21:16:56.359679+0000","last_clean":"2024-04-26T21:16:56.359679+0000","last_became_active":"2024-04-26T21:16:03.273287+0000","last_became_peered":"2024-04-26T21:16:03.273287+0000","last_unstale":"2024-04-26T21:16:56.359679+0000","last_undegraded":"2024-04-26T21:16:56.359679+0000","last_fullsized":"2024-04-26T21:16:56.359679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:48.713646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361559+0000","last_change":"2024-04-26T21:16:03.696760+0000","last_active":"2024-04-26T21:16:56.361559+0000","last_peered":"2024-04-26T21:16:56.361559+0000","last_clean":"2024-04-26T21:16:56.361559+0000","last_became_active":"2024-04-26T21:16:03.696446+0000","last_became_peered":"2024-04-26T21:16:03.696446+0000","last_unstale":"2024-04-26T21:16:56.361559+0000","last_undegraded":"2024-04-26T21:16:56.361559+0000","last_fullsized":"2024-04-26T21:16:56.361559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:30.710360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361962+0000","last_change":"2024-04-26T21:16:03.282643+0000","last_active":"2024-04-26T21:16:56.361962+0000","last_peered":"2024-04-26T21:16:56.361962+0000","last_clean":"2024-04-26T21:16:56.361962+0000","last_became_active":"2024-04-26T21:16:03.282315+0000","last_became_peered":"2024-04-26T21:16:03.282315+0000","last_unstale":"2024-04-26T21:16:56.361962+0000","last_undegraded":"2024-04-26T21:16:56.361962+0000","last_fullsized":"2024-04-26T21:16:56.361962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:13.358002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362813+0000","last_change":"2024-04-26T21:16:03.711992+0000","last_active":"2024-04-26T21:16:56.362813+0000","last_peered":"2024-04-26T21:16:56.362813+0000","last_clean":"2024-04-26T21:16:56.362813+0000","last_became_active":"2024-04-26T21:16:03.711755+0000","last_became_peered":"2024-04-26T21:16:03.711755+0000","last_unstale":"2024-04-26T21:16:56.362813+0000","last_undegraded":"2024-04-26T21:16:56.362813+0000","last_fullsized":"2024-04-26T21:16:56.362813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:01.321530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362836+0000","last_change":"2024-04-26T21:16:03.274722+0000","last_active":"2024-04-26T21:16:56.362836+0000","last_peered":"2024-04-26T21:16:56.362836+0000","last_clean":"2024-04-26T21:16:56.362836+0000","last_became_active":"2024-04-26T21:16:03.274399+0000","last_became_peered":"2024-04-26T21:16:03.274399+0000","last_unstale":"2024-04-26T21:16:56.362836+0000","last_undegraded":"2024-04-26T21:16:56.362836+0000","last_fullsized":"2024-04-26T21:16:56.362836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:35.247456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359442+0000","last_change":"2024-04-26T21:16:03.698344+0000","last_active":"2024-04-26T21:16:56.359442+0000","last_peered":"2024-04-26T21:16:56.359442+0000","last_clean":"2024-04-26T21:16:56.359442+0000","last_became_active":"2024-04-26T21:16:03.698037+0000","last_became_peered":"2024-04-26T21:16:03.698037+0000","last_unstale":"2024-04-26T21:16:56.359442+0000","last_undegraded":"2024-04-26T21:16:56.359442+0000","last_fullsized":"2024-04-26T21:16:56.359442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.573421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362176+0000","last_change":"2024-04-26T21:16:03.276459+0000","last_active":"2024-04-26T21:16:56.362176+0000","last_peered":"2024-04-26T21:16:56.362176+0000","last_clean":"2024-04-26T21:16:56.362176+0000","last_became_active":"2024-04-26T21:16:03.276108+0000","last_became_peered":"2024-04-26T21:16:03.276108+0000","last_unstale":"2024-04-26T21:16:56.362176+0000","last_undegraded":"2024-04-26T21:16:56.362176+0000","last_fullsized":"2024-04-26T21:16:56.362176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.364925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712741+0000","last_change":"2024-04-26T21:16:03.696030+0000","last_active":"2024-04-26T21:16:56.712741+0000","last_peered":"2024-04-26T21:16:56.712741+0000","last_clean":"2024-04-26T21:16:56.712741+0000","last_became_active":"2024-04-26T21:16:03.695717+0000","last_became_peered":"2024-04-26T21:16:03.695717+0000","last_unstale":"2024-04-26T21:16:56.712741+0000","last_undegraded":"2024-04-26T21:16:56.712741+0000","last_fullsized":"2024-04-26T21:16:56.712741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:55.702216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361784+0000","last_change":"2024-04-26T21:16:03.269188+0000","last_active":"2024-04-26T21:16:56.361784+0000","last_peered":"2024-04-26T21:16:56.361784+0000","last_clean":"2024-04-26T21:16:56.361784+0000","last_became_active":"2024-04-26T21:16:03.268850+0000","last_became_peered":"2024-04-26T21:16:03.268850+0000","last_unstale":"2024-04-26T21:16:56.361784+0000","last_undegraded":"2024-04-26T21:16:56.361784+0000","last_fullsized":"2024-04-26T21:16:56.361784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:54.183612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713804+0000","last_change":"2024-04-26T21:16:03.700284+0000","last_active":"2024-04-26T21:16:56.713804+0000","last_peered":"2024-04-26T21:16:56.713804+0000","last_clean":"2024-04-26T21:16:56.713804+0000","last_became_active":"2024-04-26T21:16:03.699965+0000","last_became_peered":"2024-04-26T21:16:03.699965+0000","last_unstale":"2024-04-26T21:16:56.713804+0000","last_undegraded":"2024-04-26T21:16:56.713804+0000","last_fullsized":"2024-04-26T21:16:56.713804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:40.547296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362469+0000","last_change":"2024-04-26T21:16:03.701370+0000","last_active":"2024-04-26T21:16:56.362469+0000","last_peered":"2024-04-26T21:16:56.362469+0000","last_clean":"2024-04-26T21:16:56.362469+0000","last_became_active":"2024-04-26T21:16:03.700585+0000","last_became_peered":"2024-04-26T21:16:03.700585+0000","last_unstale":"2024-04-26T21:16:56.362469+0000","last_undegraded":"2024-04-26T21:16:56.362469+0000","last_fullsized":"2024-04-26T21:16:56.362469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.348009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360613+0000","last_change":"2024-04-26T21:16:03.281141+0000","last_active":"2024-04-26T21:16:56.360613+0000","last_peered":"2024-04-26T21:16:56.360613+0000","last_clean":"2024-04-26T21:16:56.360613+0000","last_became_active":"2024-04-26T21:16:03.280812+0000","last_became_peered":"2024-04-26T21:16:03.280812+0000","last_unstale":"2024-04-26T21:16:56.360613+0000","last_undegraded":"2024-04-26T21:16:56.360613+0000","last_fullsized":"2024-04-26T21:16:56.360613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:11.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362156+0000","last_change":"2024-04-26T21:16:03.711312+0000","last_active":"2024-04-26T21:16:56.362156+0000","last_peered":"2024-04-26T21:16:56.362156+0000","last_clean":"2024-04-26T21:16:56.362156+0000","last_became_active":"2024-04-26T21:16:03.711026+0000","last_became_peered":"2024-04-26T21:16:03.711026+0000","last_unstale":"2024-04-26T21:16:56.362156+0000","last_undegraded":"2024-04-26T21:16:56.362156+0000","last_fullsized":"2024-04-26T21:16:56.362156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:10.185877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359923+0000","last_change":"2024-04-26T21:16:03.273428+0000","last_active":"2024-04-26T21:16:56.359923+0000","last_peered":"2024-04-26T21:16:56.359923+0000","last_clean":"2024-04-26T21:16:56.359923+0000","last_became_active":"2024-04-26T21:16:03.272854+0000","last_became_peered":"2024-04-26T21:16:03.272854+0000","last_unstale":"2024-04-26T21:16:56.359923+0000","last_undegraded":"2024-04-26T21:16:56.359923+0000","last_fullsized":"2024-04-26T21:16:56.359923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:48.735059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361752+0000","last_change":"2024-04-26T21:14:50.729892+0000","last_active":"2024-04-26T21:16:56.361752+0000","last_peered":"2024-04-26T21:16:56.361752+0000","last_clean":"2024-04-26T21:16:56.361752+0000","last_became_active":"2024-04-26T21:14:50.729430+0000","last_became_peered":"2024-04-26T21:14:50.729430+0000","last_unstale":"2024-04-26T21:16:56.361752+0000","last_undegraded":"2024-04-26T21:16:56.361752+0000","last_fullsized":"2024-04-26T21:16:56.361752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_clean_scrub_stamp":"2024-04-26T21:11:22.039289+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:13.510971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360509+0000","last_change":"2024-04-26T21:16:03.700702+0000","last_active":"2024-04-26T21:16:56.360509+0000","last_peered":"2024-04-26T21:16:56.360509+0000","last_clean":"2024-04-26T21:16:56.360509+0000","last_became_active":"2024-04-26T21:16:03.700412+0000","last_became_peered":"2024-04-26T21:16:03.700412+0000","last_unstale":"2024-04-26T21:16:56.360509+0000","last_undegraded":"2024-04-26T21:16:56.360509+0000","last_fullsized":"2024-04-26T21:16:56.360509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:49.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361896+0000","last_change":"2024-04-26T21:16:03.271730+0000","last_active":"2024-04-26T21:16:56.361896+0000","last_peered":"2024-04-26T21:16:56.361896+0000","last_clean":"2024-04-26T21:16:56.361896+0000","last_became_active":"2024-04-26T21:16:03.271401+0000","last_became_peered":"2024-04-26T21:16:03.271401+0000","last_unstale":"2024-04-26T21:16:56.361896+0000","last_undegraded":"2024-04-26T21:16:56.361896+0000","last_fullsized":"2024-04-26T21:16:56.361896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:16.469022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362223+0000","last_change":"2024-04-26T21:16:03.276229+0000","last_active":"2024-04-26T21:16:56.362223+0000","last_peered":"2024-04-26T21:16:56.362223+0000","last_clean":"2024-04-26T21:16:56.362223+0000","last_became_active":"2024-04-26T21:16:03.275910+0000","last_became_peered":"2024-04-26T21:16:03.275910+0000","last_unstale":"2024-04-26T21:16:56.362223+0000","last_undegraded":"2024-04-26T21:16:56.362223+0000","last_fullsized":"2024-04-26T21:16:56.362223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:03.932254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361647+0000","last_change":"2024-04-26T21:16:03.698293+0000","last_active":"2024-04-26T21:16:56.361647+0000","last_peered":"2024-04-26T21:16:56.361647+0000","last_clean":"2024-04-26T21:16:56.361647+0000","last_became_active":"2024-04-26T21:16:03.697935+0000","last_became_peered":"2024-04-26T21:16:03.697935+0000","last_unstale":"2024-04-26T21:16:56.361647+0000","last_undegraded":"2024-04-26T21:16:56.361647+0000","last_fullsized":"2024-04-26T21:16:56.361647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:54.292921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362638+0000","last_change":"2024-04-26T21:16:03.275162+0000","last_active":"2024-04-26T21:16:56.362638+0000","last_peered":"2024-04-26T21:16:56.362638+0000","last_clean":"2024-04-26T21:16:56.362638+0000","last_became_active":"2024-04-26T21:16:03.274791+0000","last_became_peered":"2024-04-26T21:16:03.274791+0000","last_unstale":"2024-04-26T21:16:56.362638+0000","last_undegraded":"2024-04-26T21:16:56.362638+0000","last_fullsized":"2024-04-26T21:16:56.362638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:15.583747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364592+0000","last_change":"2024-04-26T21:16:03.701473+0000","last_active":"2024-04-26T21:16:56.364592+0000","last_peered":"2024-04-26T21:16:56.364592+0000","last_clean":"2024-04-26T21:16:56.364592+0000","last_became_active":"2024-04-26T21:16:03.701070+0000","last_became_peered":"2024-04-26T21:16:03.701070+0000","last_unstale":"2024-04-26T21:16:56.364592+0000","last_undegraded":"2024-04-26T21:16:56.364592+0000","last_fullsized":"2024-04-26T21:16:56.364592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:25.373301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360117+0000","last_change":"2024-04-26T21:16:03.271036+0000","last_active":"2024-04-26T21:16:56.360117+0000","last_peered":"2024-04-26T21:16:56.360117+0000","last_clean":"2024-04-26T21:16:56.360117+0000","last_became_active":"2024-04-26T21:16:03.270686+0000","last_became_peered":"2024-04-26T21:16:03.270686+0000","last_unstale":"2024-04-26T21:16:56.360117+0000","last_undegraded":"2024-04-26T21:16:56.360117+0000","last_fullsized":"2024-04-26T21:16:56.360117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:21.025757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368114+0000","last_change":"2024-04-26T21:16:03.705314+0000","last_active":"2024-04-26T21:16:56.368114+0000","last_peered":"2024-04-26T21:16:56.368114+0000","last_clean":"2024-04-26T21:16:56.368114+0000","last_became_active":"2024-04-26T21:16:03.704952+0000","last_became_peered":"2024-04-26T21:16:03.704952+0000","last_unstale":"2024-04-26T21:16:56.368114+0000","last_undegraded":"2024-04-26T21:16:56.368114+0000","last_fullsized":"2024-04-26T21:16:56.368114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:57.363909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713329+0000","last_change":"2024-04-26T21:16:03.280995+0000","last_active":"2024-04-26T21:16:56.713329+0000","last_peered":"2024-04-26T21:16:56.713329+0000","last_clean":"2024-04-26T21:16:56.713329+0000","last_became_active":"2024-04-26T21:16:03.280615+0000","last_became_peered":"2024-04-26T21:16:03.280615+0000","last_unstale":"2024-04-26T21:16:56.713329+0000","last_undegraded":"2024-04-26T21:16:56.713329+0000","last_fullsized":"2024-04-26T21:16:56.713329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:32.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362097+0000","last_change":"2024-04-26T21:16:03.701218+0000","last_active":"2024-04-26T21:16:56.362097+0000","last_peered":"2024-04-26T21:16:56.362097+0000","last_clean":"2024-04-26T21:16:56.362097+0000","last_became_active":"2024-04-26T21:16:03.700895+0000","last_became_peered":"2024-04-26T21:16:03.700895+0000","last_unstale":"2024-04-26T21:16:56.362097+0000","last_undegraded":"2024-04-26T21:16:56.362097+0000","last_fullsized":"2024-04-26T21:16:56.362097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:44.104683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362857+0000","last_change":"2024-04-26T21:16:03.277235+0000","last_active":"2024-04-26T21:16:56.362857+0000","last_peered":"2024-04-26T21:16:56.362857+0000","last_clean":"2024-04-26T21:16:56.362857+0000","last_became_active":"2024-04-26T21:16:03.276901+0000","last_became_peered":"2024-04-26T21:16:03.276901+0000","last_unstale":"2024-04-26T21:16:56.362857+0000","last_undegraded":"2024-04-26T21:16:56.362857+0000","last_fullsized":"2024-04-26T21:16:56.362857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:16.297883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359135+0000","last_change":"2024-04-26T21:16:03.704318+0000","last_active":"2024-04-26T21:16:56.359135+0000","last_peered":"2024-04-26T21:16:56.359135+0000","last_clean":"2024-04-26T21:16:56.359135+0000","last_became_active":"2024-04-26T21:16:03.703907+0000","last_became_peered":"2024-04-26T21:16:03.703907+0000","last_unstale":"2024-04-26T21:16:56.359135+0000","last_undegraded":"2024-04-26T21:16:56.359135+0000","last_fullsized":"2024-04-26T21:16:56.359135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:16.431246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372105+0000","last_change":"2024-04-26T21:16:56.371135+0000","last_active":"2024-04-26T21:16:56.372105+0000","last_peered":"2024-04-26T21:16:56.372105+0000","last_clean":"2024-04-26T21:16:56.372105+0000","last_became_active":"2024-04-26T21:16:56.370666+0000","last_became_peered":"2024-04-26T21:16:56.370666+0000","last_unstale":"2024-04-26T21:16:56.372105+0000","last_undegraded":"2024-04-26T21:16:56.372105+0000","last_fullsized":"2024-04-26T21:16:56.372105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:05.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362114+0000","last_change":"2024-04-26T21:16:03.705026+0000","last_active":"2024-04-26T21:16:56.362114+0000","last_peered":"2024-04-26T21:16:56.362114+0000","last_clean":"2024-04-26T21:16:56.362114+0000","last_became_active":"2024-04-26T21:16:03.703336+0000","last_became_peered":"2024-04-26T21:16:03.703336+0000","last_unstale":"2024-04-26T21:16:56.362114+0000","last_undegraded":"2024-04-26T21:16:56.362114+0000","last_fullsized":"2024-04-26T21:16:56.362114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:11.746362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362335+0000","last_change":"2024-04-26T21:16:03.279263+0000","last_active":"2024-04-26T21:16:56.362335+0000","last_peered":"2024-04-26T21:16:56.362335+0000","last_clean":"2024-04-26T21:16:56.362335+0000","last_became_active":"2024-04-26T21:16:03.278948+0000","last_became_peered":"2024-04-26T21:16:03.278948+0000","last_unstale":"2024-04-26T21:16:56.362335+0000","last_undegraded":"2024-04-26T21:16:56.362335+0000","last_fullsized":"2024-04-26T21:16:56.362335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:39.078346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362036+0000","last_change":"2024-04-26T21:16:03.702913+0000","last_active":"2024-04-26T21:16:56.362036+0000","last_peered":"2024-04-26T21:16:56.362036+0000","last_clean":"2024-04-26T21:16:56.362036+0000","last_became_active":"2024-04-26T21:16:03.702507+0000","last_became_peered":"2024-04-26T21:16:03.702507+0000","last_unstale":"2024-04-26T21:16:56.362036+0000","last_undegraded":"2024-04-26T21:16:56.362036+0000","last_fullsized":"2024-04-26T21:16:56.362036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:51.846983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364220+0000","last_change":"2024-04-26T21:16:03.281898+0000","last_active":"2024-04-26T21:16:56.364220+0000","last_peered":"2024-04-26T21:16:56.364220+0000","last_clean":"2024-04-26T21:16:56.364220+0000","last_became_active":"2024-04-26T21:16:03.281544+0000","last_became_peered":"2024-04-26T21:16:03.281544+0000","last_unstale":"2024-04-26T21:16:56.364220+0000","last_undegraded":"2024-04-26T21:16:56.364220+0000","last_fullsized":"2024-04-26T21:16:56.364220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:39.061050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362089+0000","last_change":"2024-04-26T21:16:03.703006+0000","last_active":"2024-04-26T21:16:56.362089+0000","last_peered":"2024-04-26T21:16:56.362089+0000","last_clean":"2024-04-26T21:16:56.362089+0000","last_became_active":"2024-04-26T21:16:03.702705+0000","last_became_peered":"2024-04-26T21:16:03.702705+0000","last_unstale":"2024-04-26T21:16:56.362089+0000","last_undegraded":"2024-04-26T21:16:56.362089+0000","last_fullsized":"2024-04-26T21:16:56.362089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:11.885993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364037+0000","last_change":"2024-04-26T21:16:03.280891+0000","last_active":"2024-04-26T21:16:56.364037+0000","last_peered":"2024-04-26T21:16:56.364037+0000","last_clean":"2024-04-26T21:16:56.364037+0000","last_became_active":"2024-04-26T21:16:03.280577+0000","last_became_peered":"2024-04-26T21:16:03.280577+0000","last_unstale":"2024-04-26T21:16:56.364037+0000","last_undegraded":"2024-04-26T21:16:56.364037+0000","last_fullsized":"2024-04-26T21:16:56.364037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:28.698997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.378258+0000","last_change":"2024-04-26T21:16:56.378258+0000","last_active":"2024-04-26T21:16:56.378258+0000","last_peered":"2024-04-26T21:16:56.378258+0000","last_clean":"2024-04-26T21:16:56.378258+0000","last_became_active":"2024-04-26T21:16:56.371874+0000","last_became_peered":"2024-04-26T21:16:56.371874+0000","last_unstale":"2024-04-26T21:16:56.378258+0000","last_undegraded":"2024-04-26T21:16:56.378258+0000","last_fullsized":"2024-04-26T21:16:56.378258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:56.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360941+0000","last_change":"2024-04-26T21:16:03.276651+0000","last_active":"2024-04-26T21:16:56.360941+0000","last_peered":"2024-04-26T21:16:56.360941+0000","last_clean":"2024-04-26T21:16:56.360941+0000","last_became_active":"2024-04-26T21:16:03.276347+0000","last_became_peered":"2024-04-26T21:16:03.276347+0000","last_unstale":"2024-04-26T21:16:56.360941+0000","last_undegraded":"2024-04-26T21:16:56.360941+0000","last_fullsized":"2024-04-26T21:16:56.360941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:48.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362100+0000","last_change":"2024-04-26T21:16:03.705647+0000","last_active":"2024-04-26T21:16:56.362100+0000","last_peered":"2024-04-26T21:16:56.362100+0000","last_clean":"2024-04-26T21:16:56.362100+0000","last_became_active":"2024-04-26T21:16:03.705251+0000","last_became_peered":"2024-04-26T21:16:03.705251+0000","last_unstale":"2024-04-26T21:16:56.362100+0000","last_undegraded":"2024-04-26T21:16:56.362100+0000","last_fullsized":"2024-04-26T21:16:56.362100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:17.427221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359241+0000","last_change":"2024-04-26T21:16:03.701759+0000","last_active":"2024-04-26T21:16:56.359241+0000","last_peered":"2024-04-26T21:16:56.359241+0000","last_clean":"2024-04-26T21:16:56.359241+0000","last_became_active":"2024-04-26T21:16:03.701382+0000","last_became_peered":"2024-04-26T21:16:03.701382+0000","last_unstale":"2024-04-26T21:16:56.359241+0000","last_undegraded":"2024-04-26T21:16:56.359241+0000","last_fullsized":"2024-04-26T21:16:56.359241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.789455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372345+0000","last_change":"2024-04-26T21:16:56.372345+0000","last_active":"2024-04-26T21:16:56.372345+0000","last_peered":"2024-04-26T21:16:56.372345+0000","last_clean":"2024-04-26T21:16:56.372345+0000","last_became_active":"2024-04-26T21:16:56.371900+0000","last_became_peered":"2024-04-26T21:16:56.371900+0000","last_unstale":"2024-04-26T21:16:56.372345+0000","last_undegraded":"2024-04-26T21:16:56.372345+0000","last_fullsized":"2024-04-26T21:16:56.372345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:42.192142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361562+0000","last_change":"2024-04-26T21:16:03.702012+0000","last_active":"2024-04-26T21:16:56.361562+0000","last_peered":"2024-04-26T21:16:56.361562+0000","last_clean":"2024-04-26T21:16:56.361562+0000","last_became_active":"2024-04-26T21:16:03.701066+0000","last_became_peered":"2024-04-26T21:16:03.701066+0000","last_unstale":"2024-04-26T21:16:56.361562+0000","last_undegraded":"2024-04-26T21:16:56.361562+0000","last_fullsized":"2024-04-26T21:16:56.361562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:45.337320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367378+0000","last_change":"2024-04-26T21:16:03.702640+0000","last_active":"2024-04-26T21:16:56.367378+0000","last_peered":"2024-04-26T21:16:56.367378+0000","last_clean":"2024-04-26T21:16:56.367378+0000","last_became_active":"2024-04-26T21:16:03.702339+0000","last_became_peered":"2024-04-26T21:16:03.702339+0000","last_unstale":"2024-04-26T21:16:56.367378+0000","last_undegraded":"2024-04-26T21:16:56.367378+0000","last_fullsized":"2024-04-26T21:16:56.367378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:24.083734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361731+0000","last_change":"2024-04-26T21:16:03.708369+0000","last_active":"2024-04-26T21:16:56.361731+0000","last_peered":"2024-04-26T21:16:56.361731+0000","last_clean":"2024-04-26T21:16:56.361731+0000","last_became_active":"2024-04-26T21:16:03.708050+0000","last_became_peered":"2024-04-26T21:16:03.708050+0000","last_unstale":"2024-04-26T21:16:56.361731+0000","last_undegraded":"2024-04-26T21:16:56.361731+0000","last_fullsized":"2024-04-26T21:16:56.361731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:58.200043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359681+0000","last_change":"2024-04-26T21:16:03.700362+0000","last_active":"2024-04-26T21:16:56.359681+0000","last_peered":"2024-04-26T21:16:56.359681+0000","last_clean":"2024-04-26T21:16:56.359681+0000","last_became_active":"2024-04-26T21:16:03.700053+0000","last_became_peered":"2024-04-26T21:16:03.700053+0000","last_unstale":"2024-04-26T21:16:56.359681+0000","last_undegraded":"2024-04-26T21:16:56.359681+0000","last_fullsized":"2024-04-26T21:16:56.359681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:23.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363986+0000","last_change":"2024-04-26T21:16:03.707717+0000","last_active":"2024-04-26T21:16:56.363986+0000","last_peered":"2024-04-26T21:16:56.363986+0000","last_clean":"2024-04-26T21:16:56.363986+0000","last_became_active":"2024-04-26T21:16:03.707343+0000","last_became_peered":"2024-04-26T21:16:03.707343+0000","last_unstale":"2024-04-26T21:16:56.363986+0000","last_undegraded":"2024-04-26T21:16:56.363986+0000","last_fullsized":"2024-04-26T21:16:56.363986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:46.261231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369892+0000","last_change":"2024-04-26T21:16:56.369892+0000","last_active":"2024-04-26T21:16:56.369892+0000","last_peered":"2024-04-26T21:16:56.369892+0000","last_clean":"2024-04-26T21:16:56.369892+0000","last_became_active":"2024-04-26T21:16:56.369416+0000","last_became_peered":"2024-04-26T21:16:56.369416+0000","last_unstale":"2024-04-26T21:16:56.369892+0000","last_undegraded":"2024-04-26T21:16:56.369892+0000","last_fullsized":"2024-04-26T21:16:56.369892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:45.708969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362186+0000","last_change":"2024-04-26T21:16:03.699537+0000","last_active":"2024-04-26T21:16:56.362186+0000","last_peered":"2024-04-26T21:16:56.362186+0000","last_clean":"2024-04-26T21:16:56.362186+0000","last_became_active":"2024-04-26T21:16:03.699186+0000","last_became_peered":"2024-04-26T21:16:03.699186+0000","last_unstale":"2024-04-26T21:16:56.362186+0000","last_undegraded":"2024-04-26T21:16:56.362186+0000","last_fullsized":"2024-04-26T21:16:56.362186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:34.784929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362274+0000","last_change":"2024-04-26T21:16:03.709940+0000","last_active":"2024-04-26T21:16:56.362274+0000","last_peered":"2024-04-26T21:16:56.362274+0000","last_clean":"2024-04-26T21:16:56.362274+0000","last_became_active":"2024-04-26T21:16:03.707255+0000","last_became_peered":"2024-04-26T21:16:03.707255+0000","last_unstale":"2024-04-26T21:16:56.362274+0000","last_undegraded":"2024-04-26T21:16:56.362274+0000","last_fullsized":"2024-04-26T21:16:56.362274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:57.652194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713080+0000","last_change":"2024-04-26T21:16:03.711139+0000","last_active":"2024-04-26T21:16:56.713080+0000","last_peered":"2024-04-26T21:16:56.713080+0000","last_clean":"2024-04-26T21:16:56.713080+0000","last_became_active":"2024-04-26T21:16:03.709720+0000","last_became_peered":"2024-04-26T21:16:03.709720+0000","last_unstale":"2024-04-26T21:16:56.713080+0000","last_undegraded":"2024-04-26T21:16:56.713080+0000","last_fullsized":"2024-04-26T21:16:56.713080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:10.021914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361384+0000","last_change":"2024-04-26T21:16:03.699140+0000","last_active":"2024-04-26T21:16:56.361384+0000","last_peered":"2024-04-26T21:16:56.361384+0000","last_clean":"2024-04-26T21:16:56.361384+0000","last_became_active":"2024-04-26T21:16:03.698824+0000","last_became_peered":"2024-04-26T21:16:03.698824+0000","last_unstale":"2024-04-26T21:16:56.361384+0000","last_undegraded":"2024-04-26T21:16:56.361384+0000","last_fullsized":"2024-04-26T21:16:56.361384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:55.896278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359230+0000","last_change":"2024-04-26T21:16:03.708258+0000","last_active":"2024-04-26T21:16:56.359230+0000","last_peered":"2024-04-26T21:16:56.359230+0000","last_clean":"2024-04-26T21:16:56.359230+0000","last_became_active":"2024-04-26T21:16:03.707932+0000","last_became_peered":"2024-04-26T21:16:03.707932+0000","last_unstale":"2024-04-26T21:16:56.359230+0000","last_undegraded":"2024-04-26T21:16:56.359230+0000","last_fullsized":"2024-04-26T21:16:56.359230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:56.758307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244196,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53330,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":243,"num_read_kb":175,"num_write":77,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":160298,"data_compressed":5568,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":458793,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":1102004,"data_compressed":43255,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459492,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459397,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1049860,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":458910,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":511325,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":458883,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":511667,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":459014,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":458815,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1049460,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":52532,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52874,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":52726,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:18:07.651 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:18:07.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:07 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1438847697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:18:07.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:07 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/4057904343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:18:07.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:07 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/415082057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:18:07.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:07 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2583406038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:18:07.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.785+0000 7f70dc49c640 1 -- 172.21.15.129:0/4240463798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 msgr2=0x7f70d4101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.785+0000 7f70dc49c640 1 --2- 172.21.15.129:0/4240463798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 0x7f70d4101c90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009900 tx=0x7f70d002a410 comp rx=0 tx=0).stop 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 -- 172.21.15.129:0/4240463798 shutdown_connections 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 --2- 172.21.15.129:0/4240463798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d4102a60 0x7f70d4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 --2- 172.21.15.129:0/4240463798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4103420 0x7f70d4109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 --2- 172.21.15.129:0/4240463798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 0x7f70d4101c90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 -- 172.21.15.129:0/4240463798 >> 172.21.15.129:0/4240463798 conn(0x7f70d40fcde0 msgr2=0x7f70d40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 -- 172.21.15.129:0/4240463798 shutdown_connections 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 -- 172.21.15.129:0/4240463798 wait complete. 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 Processor -- start 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 -- start start 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 0x7f70d406a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d4102a60 0x7f70d419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4103420 0x7f70d419ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70d410d3a0 con 0x7f70d4101870 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70d410d220 con 0x7f70d4102a60 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70dc49c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f70d410d520 con 0x7f70d4103420 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70da238640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 0x7f70d406a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70da238640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 0x7f70d406a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59884/0 (socket says 172.21.15.129:59884) 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.786+0000 7f70da238640 1 -- 172.21.15.129:0/2313587924 learned_addr learned my addr 172.21.15.129:0/2313587924 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70d9a37640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d4102a60 0x7f70d419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70daa39640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4103420 0x7f70d419ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70da238640 1 -- 172.21.15.129:0/2313587924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4103420 msgr2=0x7f70d419ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70da238640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4103420 0x7f70d419ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70da238640 1 -- 172.21.15.129:0/2313587924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d4102a60 msgr2=0x7f70d419d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70da238640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d4102a60 0x7f70d419d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70da238640 1 -- 172.21.15.129:0/2313587924 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d41a0810 con 0x7f70d4101870 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70da238640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 0x7f70d406a690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f70d0006010 tx=0x7f70d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70cb7fe640 1 -- 172.21.15.129:0/2313587924 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d003f030 con 0x7f70d4101870 2024-04-26T21:18:07.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70cb7fe640 1 -- 172.21.15.129:0/2313587924 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70d0003cb0 con 0x7f70d4101870 2024-04-26T21:18:07.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70cb7fe640 1 -- 172.21.15.129:0/2313587924 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0035070 con 0x7f70d4101870 2024-04-26T21:18:07.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d41a0aa0 con 0x7f70d4101870 2024-04-26T21:18:07.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.787+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d41a0f90 con 0x7f70d4101870 2024-04-26T21:18:07.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.788+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d4108c60 con 0x7f70d4101870 2024-04-26T21:18:07.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.789+0000 7f70cb7fe640 1 -- 172.21.15.129:0/2313587924 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f70d0042030 con 0x7f70d4101870 2024-04-26T21:18:07.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.790+0000 7f70cb7fe640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70c00a3c40 0x7f70c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:07.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.790+0000 7f70cb7fe640 1 -- 172.21.15.129:0/2313587924 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f70d00f1040 con 0x7f70d4101870 2024-04-26T21:18:07.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.790+0000 7f70d9a37640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70c00a3c40 0x7f70c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:07.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.790+0000 7f70d9a37640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70c00a3c40 0x7f70c00a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f70c4005fd0 tx=0x7f70c4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:07.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.793+0000 7f70cb7fe640 1 -- 172.21.15.129:0/2313587924 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f70d00bd8f0 con 0x7f70d4101870 2024-04-26T21:18:07.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:07 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1438847697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:18:07.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:07 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/4057904343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:18:07.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:07 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/415082057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:18:07.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:07 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2583406038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:18:07.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.967+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70d4062d70 con 0x7f70c00a3c40 2024-04-26T21:18:07.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.975+0000 7f70cb7fe640 1 -- 172.21.15.129:0/2313587924 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7f70d4062d70 con 0x7f70c00a3c40 2024-04-26T21:18:07.979 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:07.980 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-26T21:18:07.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70c00a3c40 msgr2=0x7f70c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70c00a3c40 0x7f70c00a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f70c4005fd0 tx=0x7f70c4005850 comp rx=0 tx=0).stop 2024-04-26T21:18:07.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 msgr2=0x7f70d406a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:07.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 0x7f70d406a690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f70d0006010 tx=0x7f70d00042f0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 shutdown_connections 2024-04-26T21:18:07.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70c00a3c40 0x7f70c00a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d4102a60 0x7f70d419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4103420 0x7f70d419ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 --2- 172.21.15.129:0/2313587924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4101870 0x7f70d406a690 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:07.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 >> 172.21.15.129:0/2313587924 conn(0x7f70d40fcde0 msgr2=0x7f70d4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:07.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 shutdown_connections 2024-04-26T21:18:07.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:07.979+0000 7f70dc49c640 1 -- 172.21.15.129:0/2313587924 wait complete. 2024-04-26T21:18:07.991 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-04-26T21:18:07.308293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643932,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":369,"num_read_kb":284,"num_write":260,"num_write_kb":3015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360696,"kb_used_data":11484,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581128,"statfs":{"total":1151940427776,"available":1151571075072,"internally_reserved":0,"allocated":11759616,"data_stored":7437620,"data_compressed":182064,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002752"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359578+0000","last_change":"2024-04-26T21:16:03.700581+0000","last_active":"2024-04-26T21:16:56.359578+0000","last_peered":"2024-04-26T21:16:56.359578+0000","last_clean":"2024-04-26T21:16:56.359578+0000","last_became_active":"2024-04-26T21:16:03.700280+0000","last_became_peered":"2024-04-26T21:16:03.700280+0000","last_unstale":"2024-04-26T21:16:56.359578+0000","last_undegraded":"2024-04-26T21:16:56.359578+0000","last_fullsized":"2024-04-26T21:16:56.359578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:01.329784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368025+0000","last_change":"2024-04-26T21:16:03.702058+0000","last_active":"2024-04-26T21:16:56.368025+0000","last_peered":"2024-04-26T21:16:56.368025+0000","last_clean":"2024-04-26T21:16:56.368025+0000","last_became_active":"2024-04-26T21:16:03.701728+0000","last_became_peered":"2024-04-26T21:16:03.701728+0000","last_unstale":"2024-04-26T21:16:56.368025+0000","last_undegraded":"2024-04-26T21:16:56.368025+0000","last_fullsized":"2024-04-26T21:16:56.368025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:12.718225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367712+0000","last_change":"2024-04-26T21:16:03.710461+0000","last_active":"2024-04-26T21:16:56.367712+0000","last_peered":"2024-04-26T21:16:56.367712+0000","last_clean":"2024-04-26T21:16:56.367712+0000","last_became_active":"2024-04-26T21:16:03.710130+0000","last_became_peered":"2024-04-26T21:16:03.710130+0000","last_unstale":"2024-04-26T21:16:56.367712+0000","last_undegraded":"2024-04-26T21:16:56.367712+0000","last_fullsized":"2024-04-26T21:16:56.367712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:21.294677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360217+0000","last_change":"2024-04-26T21:16:03.706836+0000","last_active":"2024-04-26T21:16:56.360217+0000","last_peered":"2024-04-26T21:16:56.360217+0000","last_clean":"2024-04-26T21:16:56.360217+0000","last_became_active":"2024-04-26T21:16:03.706515+0000","last_became_peered":"2024-04-26T21:16:03.706515+0000","last_unstale":"2024-04-26T21:16:56.360217+0000","last_undegraded":"2024-04-26T21:16:56.360217+0000","last_fullsized":"2024-04-26T21:16:56.360217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:56.198337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362204+0000","last_change":"2024-04-26T21:16:03.704865+0000","last_active":"2024-04-26T21:16:56.362204+0000","last_peered":"2024-04-26T21:16:56.362204+0000","last_clean":"2024-04-26T21:16:56.362204+0000","last_became_active":"2024-04-26T21:16:03.704560+0000","last_became_peered":"2024-04-26T21:16:03.704560+0000","last_unstale":"2024-04-26T21:16:56.362204+0000","last_undegraded":"2024-04-26T21:16:56.362204+0000","last_fullsized":"2024-04-26T21:16:56.362204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:51:03.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712989+0000","last_change":"2024-04-26T21:16:03.700265+0000","last_active":"2024-04-26T21:16:56.712989+0000","last_peered":"2024-04-26T21:16:56.712989+0000","last_clean":"2024-04-26T21:16:56.712989+0000","last_became_active":"2024-04-26T21:16:03.699878+0000","last_became_peered":"2024-04-26T21:16:03.699878+0000","last_unstale":"2024-04-26T21:16:56.712989+0000","last_undegraded":"2024-04-26T21:16:56.712989+0000","last_fullsized":"2024-04-26T21:16:56.712989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:12.907254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361631+0000","last_change":"2024-04-26T21:16:03.697569+0000","last_active":"2024-04-26T21:16:56.361631+0000","last_peered":"2024-04-26T21:16:56.361631+0000","last_clean":"2024-04-26T21:16:56.361631+0000","last_became_active":"2024-04-26T21:16:03.697262+0000","last_became_peered":"2024-04-26T21:16:03.697262+0000","last_unstale":"2024-04-26T21:16:56.361631+0000","last_undegraded":"2024-04-26T21:16:56.361631+0000","last_fullsized":"2024-04-26T21:16:56.361631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:28.645750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363218+0000","last_change":"2024-04-26T21:16:03.705788+0000","last_active":"2024-04-26T21:16:56.363218+0000","last_peered":"2024-04-26T21:16:56.363218+0000","last_clean":"2024-04-26T21:16:56.363218+0000","last_became_active":"2024-04-26T21:16:03.705479+0000","last_became_peered":"2024-04-26T21:16:03.705479+0000","last_unstale":"2024-04-26T21:16:56.363218+0000","last_undegraded":"2024-04-26T21:16:56.363218+0000","last_fullsized":"2024-04-26T21:16:56.363218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:27.306777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713655+0000","last_change":"2024-04-26T21:16:03.709716+0000","last_active":"2024-04-26T21:16:56.713655+0000","last_peered":"2024-04-26T21:16:56.713655+0000","last_clean":"2024-04-26T21:16:56.713655+0000","last_became_active":"2024-04-26T21:16:03.709219+0000","last_became_peered":"2024-04-26T21:16:03.709219+0000","last_unstale":"2024-04-26T21:16:56.713655+0000","last_undegraded":"2024-04-26T21:16:56.713655+0000","last_fullsized":"2024-04-26T21:16:56.713655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:18.936075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713179+0000","last_change":"2024-04-26T21:16:03.694745+0000","last_active":"2024-04-26T21:16:56.713179+0000","last_peered":"2024-04-26T21:16:56.713179+0000","last_clean":"2024-04-26T21:16:56.713179+0000","last_became_active":"2024-04-26T21:16:03.694044+0000","last_became_peered":"2024-04-26T21:16:03.694044+0000","last_unstale":"2024-04-26T21:16:56.713179+0000","last_undegraded":"2024-04-26T21:16:56.713179+0000","last_fullsized":"2024-04-26T21:16:56.713179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:13.060467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362033+0000","last_change":"2024-04-26T21:16:03.706434+0000","last_active":"2024-04-26T21:16:56.362033+0000","last_peered":"2024-04-26T21:16:56.362033+0000","last_clean":"2024-04-26T21:16:56.362033+0000","last_became_active":"2024-04-26T21:16:03.706093+0000","last_became_peered":"2024-04-26T21:16:03.706093+0000","last_unstale":"2024-04-26T21:16:56.362033+0000","last_undegraded":"2024-04-26T21:16:56.362033+0000","last_fullsized":"2024-04-26T21:16:56.362033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:04.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360084+0000","last_change":"2024-04-26T21:16:03.707672+0000","last_active":"2024-04-26T21:16:56.360084+0000","last_peered":"2024-04-26T21:16:56.360084+0000","last_clean":"2024-04-26T21:16:56.360084+0000","last_became_active":"2024-04-26T21:16:03.707344+0000","last_became_peered":"2024-04-26T21:16:03.707344+0000","last_unstale":"2024-04-26T21:16:56.360084+0000","last_undegraded":"2024-04-26T21:16:56.360084+0000","last_fullsized":"2024-04-26T21:16:56.360084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:07.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361281+0000","last_change":"2024-04-26T21:16:03.699816+0000","last_active":"2024-04-26T21:16:56.361281+0000","last_peered":"2024-04-26T21:16:56.361281+0000","last_clean":"2024-04-26T21:16:56.361281+0000","last_became_active":"2024-04-26T21:16:03.699514+0000","last_became_peered":"2024-04-26T21:16:03.699514+0000","last_unstale":"2024-04-26T21:16:56.361281+0000","last_undegraded":"2024-04-26T21:16:56.361281+0000","last_fullsized":"2024-04-26T21:16:56.361281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:59.230438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367152+0000","last_change":"2024-04-26T21:16:03.709627+0000","last_active":"2024-04-26T21:16:56.367152+0000","last_peered":"2024-04-26T21:16:56.367152+0000","last_clean":"2024-04-26T21:16:56.367152+0000","last_became_active":"2024-04-26T21:16:03.709238+0000","last_became_peered":"2024-04-26T21:16:03.709238+0000","last_unstale":"2024-04-26T21:16:56.367152+0000","last_undegraded":"2024-04-26T21:16:56.367152+0000","last_fullsized":"2024-04-26T21:16:56.367152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:55.303059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.366111+0000","last_change":"2024-04-26T21:16:03.702275+0000","last_active":"2024-04-26T21:16:56.366111+0000","last_peered":"2024-04-26T21:16:56.366111+0000","last_clean":"2024-04-26T21:16:56.366111+0000","last_became_active":"2024-04-26T21:16:03.701966+0000","last_became_peered":"2024-04-26T21:16:03.701966+0000","last_unstale":"2024-04-26T21:16:56.366111+0000","last_undegraded":"2024-04-26T21:16:56.366111+0000","last_fullsized":"2024-04-26T21:16:56.366111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:02.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713281+0000","last_change":"2024-04-26T21:16:03.711979+0000","last_active":"2024-04-26T21:16:56.713281+0000","last_peered":"2024-04-26T21:16:56.713281+0000","last_clean":"2024-04-26T21:16:56.713281+0000","last_became_active":"2024-04-26T21:16:03.711637+0000","last_became_peered":"2024-04-26T21:16:03.711637+0000","last_unstale":"2024-04-26T21:16:56.713281+0000","last_undegraded":"2024-04-26T21:16:56.713281+0000","last_fullsized":"2024-04-26T21:16:56.713281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:27.817993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359639+0000","last_change":"2024-04-26T21:16:03.701303+0000","last_active":"2024-04-26T21:16:56.359639+0000","last_peered":"2024-04-26T21:16:56.359639+0000","last_clean":"2024-04-26T21:16:56.359639+0000","last_became_active":"2024-04-26T21:16:03.701001+0000","last_became_peered":"2024-04-26T21:16:03.701001+0000","last_unstale":"2024-04-26T21:16:56.359639+0000","last_undegraded":"2024-04-26T21:16:56.359639+0000","last_fullsized":"2024-04-26T21:16:56.359639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:08.349915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362455+0000","last_change":"2024-04-26T21:16:03.696775+0000","last_active":"2024-04-26T21:16:56.362455+0000","last_peered":"2024-04-26T21:16:56.362455+0000","last_clean":"2024-04-26T21:16:56.362455+0000","last_became_active":"2024-04-26T21:16:03.696433+0000","last_became_peered":"2024-04-26T21:16:03.696433+0000","last_unstale":"2024-04-26T21:16:56.362455+0000","last_undegraded":"2024-04-26T21:16:56.362455+0000","last_fullsized":"2024-04-26T21:16:56.362455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:50.029169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361906+0000","last_change":"2024-04-26T21:16:03.698188+0000","last_active":"2024-04-26T21:16:56.361906+0000","last_peered":"2024-04-26T21:16:56.361906+0000","last_clean":"2024-04-26T21:16:56.361906+0000","last_became_active":"2024-04-26T21:16:03.697882+0000","last_became_peered":"2024-04-26T21:16:03.697882+0000","last_unstale":"2024-04-26T21:16:56.361906+0000","last_undegraded":"2024-04-26T21:16:56.361906+0000","last_fullsized":"2024-04-26T21:16:56.361906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:04.737619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362709+0000","last_change":"2024-04-26T21:16:03.275296+0000","last_active":"2024-04-26T21:16:56.362709+0000","last_peered":"2024-04-26T21:16:56.362709+0000","last_clean":"2024-04-26T21:16:56.362709+0000","last_became_active":"2024-04-26T21:16:03.274733+0000","last_became_peered":"2024-04-26T21:16:03.274733+0000","last_unstale":"2024-04-26T21:16:56.362709+0000","last_undegraded":"2024-04-26T21:16:56.362709+0000","last_fullsized":"2024-04-26T21:16:56.362709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:16.512934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363299+0000","last_change":"2024-04-26T21:16:03.705823+0000","last_active":"2024-04-26T21:16:56.363299+0000","last_peered":"2024-04-26T21:16:56.363299+0000","last_clean":"2024-04-26T21:16:56.363299+0000","last_became_active":"2024-04-26T21:16:03.705512+0000","last_became_peered":"2024-04-26T21:16:03.705512+0000","last_unstale":"2024-04-26T21:16:56.363299+0000","last_undegraded":"2024-04-26T21:16:56.363299+0000","last_fullsized":"2024-04-26T21:16:56.363299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:17.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364347+0000","last_change":"2024-04-26T21:16:03.285008+0000","last_active":"2024-04-26T21:16:56.364347+0000","last_peered":"2024-04-26T21:16:56.364347+0000","last_clean":"2024-04-26T21:16:56.364347+0000","last_became_active":"2024-04-26T21:16:03.284679+0000","last_became_peered":"2024-04-26T21:16:03.284679+0000","last_unstale":"2024-04-26T21:16:56.364347+0000","last_undegraded":"2024-04-26T21:16:56.364347+0000","last_fullsized":"2024-04-26T21:16:56.364347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:35.404086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368858+0000","last_change":"2024-04-26T21:16:03.710571+0000","last_active":"2024-04-26T21:16:56.368858+0000","last_peered":"2024-04-26T21:16:56.368858+0000","last_clean":"2024-04-26T21:16:56.368858+0000","last_became_active":"2024-04-26T21:16:03.710237+0000","last_became_peered":"2024-04-26T21:16:03.710237+0000","last_unstale":"2024-04-26T21:16:56.368858+0000","last_undegraded":"2024-04-26T21:16:56.368858+0000","last_fullsized":"2024-04-26T21:16:56.368858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:53.188554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369209+0000","last_change":"2024-04-26T21:16:56.368746+0000","last_active":"2024-04-26T21:16:56.369209+0000","last_peered":"2024-04-26T21:16:56.369209+0000","last_clean":"2024-04-26T21:16:56.369209+0000","last_became_active":"2024-04-26T21:16:56.365062+0000","last_became_peered":"2024-04-26T21:16:56.365062+0000","last_unstale":"2024-04-26T21:16:56.369209+0000","last_undegraded":"2024-04-26T21:16:56.369209+0000","last_fullsized":"2024-04-26T21:16:56.369209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:36.165146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713534+0000","last_change":"2024-04-26T21:16:03.711355+0000","last_active":"2024-04-26T21:16:56.713534+0000","last_peered":"2024-04-26T21:16:56.713534+0000","last_clean":"2024-04-26T21:16:56.713534+0000","last_became_active":"2024-04-26T21:16:03.710885+0000","last_became_peered":"2024-04-26T21:16:03.710885+0000","last_unstale":"2024-04-26T21:16:56.713534+0000","last_undegraded":"2024-04-26T21:16:56.713534+0000","last_fullsized":"2024-04-26T21:16:56.713534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:10.515590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362768+0000","last_change":"2024-04-26T21:16:03.278484+0000","last_active":"2024-04-26T21:16:56.362768+0000","last_peered":"2024-04-26T21:16:56.362768+0000","last_clean":"2024-04-26T21:16:56.362768+0000","last_became_active":"2024-04-26T21:16:03.277402+0000","last_became_peered":"2024-04-26T21:16:03.277402+0000","last_unstale":"2024-04-26T21:16:56.362768+0000","last_undegraded":"2024-04-26T21:16:56.362768+0000","last_fullsized":"2024-04-26T21:16:56.362768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:21.807942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714671+0000","last_change":"2024-04-26T21:16:56.714671+0000","last_active":"2024-04-26T21:16:56.714671+0000","last_peered":"2024-04-26T21:16:56.714671+0000","last_clean":"2024-04-26T21:16:56.714671+0000","last_became_active":"2024-04-26T21:16:56.714209+0000","last_became_peered":"2024-04-26T21:16:56.714209+0000","last_unstale":"2024-04-26T21:16:56.714671+0000","last_undegraded":"2024-04-26T21:16:56.714671+0000","last_fullsized":"2024-04-26T21:16:56.714671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:44.790915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363442+0000","last_change":"2024-04-26T21:16:03.277219+0000","last_active":"2024-04-26T21:16:56.363442+0000","last_peered":"2024-04-26T21:16:56.363442+0000","last_clean":"2024-04-26T21:16:56.363442+0000","last_became_active":"2024-04-26T21:16:03.276885+0000","last_became_peered":"2024-04-26T21:16:03.276885+0000","last_unstale":"2024-04-26T21:16:56.363442+0000","last_undegraded":"2024-04-26T21:16:56.363442+0000","last_fullsized":"2024-04-26T21:16:56.363442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:45.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362963+0000","last_change":"2024-04-26T21:16:03.703453+0000","last_active":"2024-04-26T21:16:56.362963+0000","last_peered":"2024-04-26T21:16:56.362963+0000","last_clean":"2024-04-26T21:16:56.362963+0000","last_became_active":"2024-04-26T21:16:03.703161+0000","last_became_peered":"2024-04-26T21:16:03.703161+0000","last_unstale":"2024-04-26T21:16:56.362963+0000","last_undegraded":"2024-04-26T21:16:56.362963+0000","last_fullsized":"2024-04-26T21:16:56.362963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.999243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712837+0000","last_change":"2024-04-26T21:16:03.274815+0000","last_active":"2024-04-26T21:16:56.712837+0000","last_peered":"2024-04-26T21:16:56.712837+0000","last_clean":"2024-04-26T21:16:56.712837+0000","last_became_active":"2024-04-26T21:16:03.274509+0000","last_became_peered":"2024-04-26T21:16:03.274509+0000","last_unstale":"2024-04-26T21:16:56.712837+0000","last_undegraded":"2024-04-26T21:16:56.712837+0000","last_fullsized":"2024-04-26T21:16:56.712837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:27.212714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.371100+0000","last_change":"2024-04-26T21:16:56.371100+0000","last_active":"2024-04-26T21:16:56.371100+0000","last_peered":"2024-04-26T21:16:56.371100+0000","last_clean":"2024-04-26T21:16:56.371100+0000","last_became_active":"2024-04-26T21:16:56.370711+0000","last_became_peered":"2024-04-26T21:16:56.370711+0000","last_unstale":"2024-04-26T21:16:56.371100+0000","last_undegraded":"2024-04-26T21:16:56.371100+0000","last_fullsized":"2024-04-26T21:16:56.371100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:00.143815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360698+0000","last_change":"2024-04-26T21:16:03.270417+0000","last_active":"2024-04-26T21:16:56.360698+0000","last_peered":"2024-04-26T21:16:56.360698+0000","last_clean":"2024-04-26T21:16:56.360698+0000","last_became_active":"2024-04-26T21:16:03.270066+0000","last_became_peered":"2024-04-26T21:16:03.270066+0000","last_unstale":"2024-04-26T21:16:56.360698+0000","last_undegraded":"2024-04-26T21:16:56.360698+0000","last_fullsized":"2024-04-26T21:16:56.360698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:46.794698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363117+0000","last_change":"2024-04-26T21:16:03.700221+0000","last_active":"2024-04-26T21:16:56.363117+0000","last_peered":"2024-04-26T21:16:56.363117+0000","last_clean":"2024-04-26T21:16:56.363117+0000","last_became_active":"2024-04-26T21:16:03.699798+0000","last_became_peered":"2024-04-26T21:16:03.699798+0000","last_unstale":"2024-04-26T21:16:56.363117+0000","last_undegraded":"2024-04-26T21:16:56.363117+0000","last_fullsized":"2024-04-26T21:16:56.363117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:52.083700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362074+0000","last_change":"2024-04-26T21:16:03.282355+0000","last_active":"2024-04-26T21:16:56.362074+0000","last_peered":"2024-04-26T21:16:56.362074+0000","last_clean":"2024-04-26T21:16:56.362074+0000","last_became_active":"2024-04-26T21:16:03.281169+0000","last_became_peered":"2024-04-26T21:16:03.281169+0000","last_unstale":"2024-04-26T21:16:56.362074+0000","last_undegraded":"2024-04-26T21:16:56.362074+0000","last_fullsized":"2024-04-26T21:16:56.362074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:55.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714327+0000","last_change":"2024-04-26T21:16:56.714327+0000","last_active":"2024-04-26T21:16:56.714327+0000","last_peered":"2024-04-26T21:16:56.714327+0000","last_clean":"2024-04-26T21:16:56.714327+0000","last_became_active":"2024-04-26T21:16:56.713967+0000","last_became_peered":"2024-04-26T21:16:56.713967+0000","last_unstale":"2024-04-26T21:16:56.714327+0000","last_undegraded":"2024-04-26T21:16:56.714327+0000","last_fullsized":"2024-04-26T21:16:56.714327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:32.475044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713294+0000","last_change":"2024-04-26T21:16:03.284948+0000","last_active":"2024-04-26T21:16:56.713294+0000","last_peered":"2024-04-26T21:16:56.713294+0000","last_clean":"2024-04-26T21:16:56.713294+0000","last_became_active":"2024-04-26T21:16:03.284570+0000","last_became_peered":"2024-04-26T21:16:03.284570+0000","last_unstale":"2024-04-26T21:16:56.713294+0000","last_undegraded":"2024-04-26T21:16:56.713294+0000","last_fullsized":"2024-04-26T21:16:56.713294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:22.883834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713580+0000","last_change":"2024-04-26T21:16:03.710359+0000","last_active":"2024-04-26T21:16:56.713580+0000","last_peered":"2024-04-26T21:16:56.713580+0000","last_clean":"2024-04-26T21:16:56.713580+0000","last_became_active":"2024-04-26T21:16:03.710053+0000","last_became_peered":"2024-04-26T21:16:03.710053+0000","last_unstale":"2024-04-26T21:16:56.713580+0000","last_undegraded":"2024-04-26T21:16:56.713580+0000","last_fullsized":"2024-04-26T21:16:56.713580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:45.981083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361874+0000","last_change":"2024-04-26T21:16:03.278787+0000","last_active":"2024-04-26T21:16:56.361874+0000","last_peered":"2024-04-26T21:16:56.361874+0000","last_clean":"2024-04-26T21:16:56.361874+0000","last_became_active":"2024-04-26T21:16:03.278483+0000","last_became_peered":"2024-04-26T21:16:03.278483+0000","last_unstale":"2024-04-26T21:16:56.361874+0000","last_undegraded":"2024-04-26T21:16:56.361874+0000","last_fullsized":"2024-04-26T21:16:56.361874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:40.303372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372769+0000","last_change":"2024-04-26T21:16:56.372769+0000","last_active":"2024-04-26T21:16:56.372769+0000","last_peered":"2024-04-26T21:16:56.372769+0000","last_clean":"2024-04-26T21:16:56.372769+0000","last_became_active":"2024-04-26T21:16:56.372393+0000","last_became_peered":"2024-04-26T21:16:56.372393+0000","last_unstale":"2024-04-26T21:16:56.372769+0000","last_undegraded":"2024-04-26T21:16:56.372769+0000","last_fullsized":"2024-04-26T21:16:56.372769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:29.818635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364174+0000","last_change":"2024-04-26T21:16:03.277387+0000","last_active":"2024-04-26T21:16:56.364174+0000","last_peered":"2024-04-26T21:16:56.364174+0000","last_clean":"2024-04-26T21:16:56.364174+0000","last_became_active":"2024-04-26T21:16:03.274765+0000","last_became_peered":"2024-04-26T21:16:03.274765+0000","last_unstale":"2024-04-26T21:16:56.364174+0000","last_undegraded":"2024-04-26T21:16:56.364174+0000","last_fullsized":"2024-04-26T21:16:56.364174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:51.081883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713473+0000","last_change":"2024-04-26T21:16:03.710977+0000","last_active":"2024-04-26T21:16:56.713473+0000","last_peered":"2024-04-26T21:16:56.713473+0000","last_clean":"2024-04-26T21:16:56.713473+0000","last_became_active":"2024-04-26T21:16:03.710284+0000","last_became_peered":"2024-04-26T21:16:03.710284+0000","last_unstale":"2024-04-26T21:16:56.713473+0000","last_undegraded":"2024-04-26T21:16:56.713473+0000","last_fullsized":"2024-04-26T21:16:56.713473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:46.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'25","reported_seq":281,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:18:01.869515+0000","last_change":"2024-04-26T21:16:03.276865+0000","last_active":"2024-04-26T21:18:01.869515+0000","last_peered":"2024-04-26T21:18:01.869515+0000","last_clean":"2024-04-26T21:18:01.869515+0000","last_became_active":"2024-04-26T21:16:03.276564+0000","last_became_peered":"2024-04-26T21:16:03.276564+0000","last_unstale":"2024-04-26T21:18:01.869515+0000","last_undegraded":"2024-04-26T21:18:01.869515+0000","last_fullsized":"2024-04-26T21:18:01.869515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:33.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52532,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":237,"num_read_kb":169,"num_write":25,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362088+0000","last_change":"2024-04-26T21:16:03.700441+0000","last_active":"2024-04-26T21:16:56.362088+0000","last_peered":"2024-04-26T21:16:56.362088+0000","last_clean":"2024-04-26T21:16:56.362088+0000","last_became_active":"2024-04-26T21:16:03.700142+0000","last_became_peered":"2024-04-26T21:16:03.700142+0000","last_unstale":"2024-04-26T21:16:56.362088+0000","last_undegraded":"2024-04-26T21:16:56.362088+0000","last_fullsized":"2024-04-26T21:16:56.362088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:35.166603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361872+0000","last_change":"2024-04-26T21:16:03.280144+0000","last_active":"2024-04-26T21:16:56.361872+0000","last_peered":"2024-04-26T21:16:56.361872+0000","last_clean":"2024-04-26T21:16:56.361872+0000","last_became_active":"2024-04-26T21:16:03.279844+0000","last_became_peered":"2024-04-26T21:16:03.279844+0000","last_unstale":"2024-04-26T21:16:56.361872+0000","last_undegraded":"2024-04-26T21:16:56.361872+0000","last_fullsized":"2024-04-26T21:16:56.361872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:00.704181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361493+0000","last_change":"2024-04-26T21:16:03.701732+0000","last_active":"2024-04-26T21:16:56.361493+0000","last_peered":"2024-04-26T21:16:56.361493+0000","last_clean":"2024-04-26T21:16:56.361493+0000","last_became_active":"2024-04-26T21:16:03.701424+0000","last_became_peered":"2024-04-26T21:16:03.701424+0000","last_unstale":"2024-04-26T21:16:56.361493+0000","last_undegraded":"2024-04-26T21:16:56.361493+0000","last_fullsized":"2024-04-26T21:16:56.361493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:02.293086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363077+0000","last_change":"2024-04-26T21:16:03.276816+0000","last_active":"2024-04-26T21:16:56.363077+0000","last_peered":"2024-04-26T21:16:56.363077+0000","last_clean":"2024-04-26T21:16:56.363077+0000","last_became_active":"2024-04-26T21:16:03.276471+0000","last_became_peered":"2024-04-26T21:16:03.276471+0000","last_unstale":"2024-04-26T21:16:56.363077+0000","last_undegraded":"2024-04-26T21:16:56.363077+0000","last_fullsized":"2024-04-26T21:16:56.363077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:44.630638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361866+0000","last_change":"2024-04-26T21:16:03.701002+0000","last_active":"2024-04-26T21:16:56.361866+0000","last_peered":"2024-04-26T21:16:56.361866+0000","last_clean":"2024-04-26T21:16:56.361866+0000","last_became_active":"2024-04-26T21:16:03.700649+0000","last_became_peered":"2024-04-26T21:16:03.700649+0000","last_unstale":"2024-04-26T21:16:56.361866+0000","last_undegraded":"2024-04-26T21:16:56.361866+0000","last_fullsized":"2024-04-26T21:16:56.361866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:45.571735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359679+0000","last_change":"2024-04-26T21:16:03.273727+0000","last_active":"2024-04-26T21:16:56.359679+0000","last_peered":"2024-04-26T21:16:56.359679+0000","last_clean":"2024-04-26T21:16:56.359679+0000","last_became_active":"2024-04-26T21:16:03.273287+0000","last_became_peered":"2024-04-26T21:16:03.273287+0000","last_unstale":"2024-04-26T21:16:56.359679+0000","last_undegraded":"2024-04-26T21:16:56.359679+0000","last_fullsized":"2024-04-26T21:16:56.359679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:48.713646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361559+0000","last_change":"2024-04-26T21:16:03.696760+0000","last_active":"2024-04-26T21:16:56.361559+0000","last_peered":"2024-04-26T21:16:56.361559+0000","last_clean":"2024-04-26T21:16:56.361559+0000","last_became_active":"2024-04-26T21:16:03.696446+0000","last_became_peered":"2024-04-26T21:16:03.696446+0000","last_unstale":"2024-04-26T21:16:56.361559+0000","last_undegraded":"2024-04-26T21:16:56.361559+0000","last_fullsized":"2024-04-26T21:16:56.361559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:30.710360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361962+0000","last_change":"2024-04-26T21:16:03.282643+0000","last_active":"2024-04-26T21:16:56.361962+0000","last_peered":"2024-04-26T21:16:56.361962+0000","last_clean":"2024-04-26T21:16:56.361962+0000","last_became_active":"2024-04-26T21:16:03.282315+0000","last_became_peered":"2024-04-26T21:16:03.282315+0000","last_unstale":"2024-04-26T21:16:56.361962+0000","last_undegraded":"2024-04-26T21:16:56.361962+0000","last_fullsized":"2024-04-26T21:16:56.361962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:13.358002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362813+0000","last_change":"2024-04-26T21:16:03.711992+0000","last_active":"2024-04-26T21:16:56.362813+0000","last_peered":"2024-04-26T21:16:56.362813+0000","last_clean":"2024-04-26T21:16:56.362813+0000","last_became_active":"2024-04-26T21:16:03.711755+0000","last_became_peered":"2024-04-26T21:16:03.711755+0000","last_unstale":"2024-04-26T21:16:56.362813+0000","last_undegraded":"2024-04-26T21:16:56.362813+0000","last_fullsized":"2024-04-26T21:16:56.362813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:01.321530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362836+0000","last_change":"2024-04-26T21:16:03.274722+0000","last_active":"2024-04-26T21:16:56.362836+0000","last_peered":"2024-04-26T21:16:56.362836+0000","last_clean":"2024-04-26T21:16:56.362836+0000","last_became_active":"2024-04-26T21:16:03.274399+0000","last_became_peered":"2024-04-26T21:16:03.274399+0000","last_unstale":"2024-04-26T21:16:56.362836+0000","last_undegraded":"2024-04-26T21:16:56.362836+0000","last_fullsized":"2024-04-26T21:16:56.362836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:35.247456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359442+0000","last_change":"2024-04-26T21:16:03.698344+0000","last_active":"2024-04-26T21:16:56.359442+0000","last_peered":"2024-04-26T21:16:56.359442+0000","last_clean":"2024-04-26T21:16:56.359442+0000","last_became_active":"2024-04-26T21:16:03.698037+0000","last_became_peered":"2024-04-26T21:16:03.698037+0000","last_unstale":"2024-04-26T21:16:56.359442+0000","last_undegraded":"2024-04-26T21:16:56.359442+0000","last_fullsized":"2024-04-26T21:16:56.359442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.573421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362176+0000","last_change":"2024-04-26T21:16:03.276459+0000","last_active":"2024-04-26T21:16:56.362176+0000","last_peered":"2024-04-26T21:16:56.362176+0000","last_clean":"2024-04-26T21:16:56.362176+0000","last_became_active":"2024-04-26T21:16:03.276108+0000","last_became_peered":"2024-04-26T21:16:03.276108+0000","last_unstale":"2024-04-26T21:16:56.362176+0000","last_undegraded":"2024-04-26T21:16:56.362176+0000","last_fullsized":"2024-04-26T21:16:56.362176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.364925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712741+0000","last_change":"2024-04-26T21:16:03.696030+0000","last_active":"2024-04-26T21:16:56.712741+0000","last_peered":"2024-04-26T21:16:56.712741+0000","last_clean":"2024-04-26T21:16:56.712741+0000","last_became_active":"2024-04-26T21:16:03.695717+0000","last_became_peered":"2024-04-26T21:16:03.695717+0000","last_unstale":"2024-04-26T21:16:56.712741+0000","last_undegraded":"2024-04-26T21:16:56.712741+0000","last_fullsized":"2024-04-26T21:16:56.712741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:55.702216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361784+0000","last_change":"2024-04-26T21:16:03.269188+0000","last_active":"2024-04-26T21:16:56.361784+0000","last_peered":"2024-04-26T21:16:56.361784+0000","last_clean":"2024-04-26T21:16:56.361784+0000","last_became_active":"2024-04-26T21:16:03.268850+0000","last_became_peered":"2024-04-26T21:16:03.268850+0000","last_unstale":"2024-04-26T21:16:56.361784+0000","last_undegraded":"2024-04-26T21:16:56.361784+0000","last_fullsized":"2024-04-26T21:16:56.361784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:54.183612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713804+0000","last_change":"2024-04-26T21:16:03.700284+0000","last_active":"2024-04-26T21:16:56.713804+0000","last_peered":"2024-04-26T21:16:56.713804+0000","last_clean":"2024-04-26T21:16:56.713804+0000","last_became_active":"2024-04-26T21:16:03.699965+0000","last_became_peered":"2024-04-26T21:16:03.699965+0000","last_unstale":"2024-04-26T21:16:56.713804+0000","last_undegraded":"2024-04-26T21:16:56.713804+0000","last_fullsized":"2024-04-26T21:16:56.713804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:40.547296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362469+0000","last_change":"2024-04-26T21:16:03.701370+0000","last_active":"2024-04-26T21:16:56.362469+0000","last_peered":"2024-04-26T21:16:56.362469+0000","last_clean":"2024-04-26T21:16:56.362469+0000","last_became_active":"2024-04-26T21:16:03.700585+0000","last_became_peered":"2024-04-26T21:16:03.700585+0000","last_unstale":"2024-04-26T21:16:56.362469+0000","last_undegraded":"2024-04-26T21:16:56.362469+0000","last_fullsized":"2024-04-26T21:16:56.362469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.348009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360613+0000","last_change":"2024-04-26T21:16:03.281141+0000","last_active":"2024-04-26T21:16:56.360613+0000","last_peered":"2024-04-26T21:16:56.360613+0000","last_clean":"2024-04-26T21:16:56.360613+0000","last_became_active":"2024-04-26T21:16:03.280812+0000","last_became_peered":"2024-04-26T21:16:03.280812+0000","last_unstale":"2024-04-26T21:16:56.360613+0000","last_undegraded":"2024-04-26T21:16:56.360613+0000","last_fullsized":"2024-04-26T21:16:56.360613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:11.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362156+0000","last_change":"2024-04-26T21:16:03.711312+0000","last_active":"2024-04-26T21:16:56.362156+0000","last_peered":"2024-04-26T21:16:56.362156+0000","last_clean":"2024-04-26T21:16:56.362156+0000","last_became_active":"2024-04-26T21:16:03.711026+0000","last_became_peered":"2024-04-26T21:16:03.711026+0000","last_unstale":"2024-04-26T21:16:56.362156+0000","last_undegraded":"2024-04-26T21:16:56.362156+0000","last_fullsized":"2024-04-26T21:16:56.362156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:10.185877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359923+0000","last_change":"2024-04-26T21:16:03.273428+0000","last_active":"2024-04-26T21:16:56.359923+0000","last_peered":"2024-04-26T21:16:56.359923+0000","last_clean":"2024-04-26T21:16:56.359923+0000","last_became_active":"2024-04-26T21:16:03.272854+0000","last_became_peered":"2024-04-26T21:16:03.272854+0000","last_unstale":"2024-04-26T21:16:56.359923+0000","last_undegraded":"2024-04-26T21:16:56.359923+0000","last_fullsized":"2024-04-26T21:16:56.359923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:48.735059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361752+0000","last_change":"2024-04-26T21:14:50.729892+0000","last_active":"2024-04-26T21:16:56.361752+0000","last_peered":"2024-04-26T21:16:56.361752+0000","last_clean":"2024-04-26T21:16:56.361752+0000","last_became_active":"2024-04-26T21:14:50.729430+0000","last_became_peered":"2024-04-26T21:14:50.729430+0000","last_unstale":"2024-04-26T21:16:56.361752+0000","last_undegraded":"2024-04-26T21:16:56.361752+0000","last_fullsized":"2024-04-26T21:16:56.361752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_clean_scrub_stamp":"2024-04-26T21:11:22.039289+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:13.510971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360509+0000","last_change":"2024-04-26T21:16:03.700702+0000","last_active":"2024-04-26T21:16:56.360509+0000","last_peered":"2024-04-26T21:16:56.360509+0000","last_clean":"2024-04-26T21:16:56.360509+0000","last_became_active":"2024-04-26T21:16:03.700412+0000","last_became_peered":"2024-04-26T21:16:03.700412+0000","last_unstale":"2024-04-26T21:16:56.360509+0000","last_undegraded":"2024-04-26T21:16:56.360509+0000","last_fullsized":"2024-04-26T21:16:56.360509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:49.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361896+0000","last_change":"2024-04-26T21:16:03.271730+0000","last_active":"2024-04-26T21:16:56.361896+0000","last_peered":"2024-04-26T21:16:56.361896+0000","last_clean":"2024-04-26T21:16:56.361896+0000","last_became_active":"2024-04-26T21:16:03.271401+0000","last_became_peered":"2024-04-26T21:16:03.271401+0000","last_unstale":"2024-04-26T21:16:56.361896+0000","last_undegraded":"2024-04-26T21:16:56.361896+0000","last_fullsized":"2024-04-26T21:16:56.361896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:16.469022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362223+0000","last_change":"2024-04-26T21:16:03.276229+0000","last_active":"2024-04-26T21:16:56.362223+0000","last_peered":"2024-04-26T21:16:56.362223+0000","last_clean":"2024-04-26T21:16:56.362223+0000","last_became_active":"2024-04-26T21:16:03.275910+0000","last_became_peered":"2024-04-26T21:16:03.275910+0000","last_unstale":"2024-04-26T21:16:56.362223+0000","last_undegraded":"2024-04-26T21:16:56.362223+0000","last_fullsized":"2024-04-26T21:16:56.362223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:03.932254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361647+0000","last_change":"2024-04-26T21:16:03.698293+0000","last_active":"2024-04-26T21:16:56.361647+0000","last_peered":"2024-04-26T21:16:56.361647+0000","last_clean":"2024-04-26T21:16:56.361647+0000","last_became_active":"2024-04-26T21:16:03.697935+0000","last_became_peered":"2024-04-26T21:16:03.697935+0000","last_unstale":"2024-04-26T21:16:56.361647+0000","last_undegraded":"2024-04-26T21:16:56.361647+0000","last_fullsized":"2024-04-26T21:16:56.361647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:54.292921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362638+0000","last_change":"2024-04-26T21:16:03.275162+0000","last_active":"2024-04-26T21:16:56.362638+0000","last_peered":"2024-04-26T21:16:56.362638+0000","last_clean":"2024-04-26T21:16:56.362638+0000","last_became_active":"2024-04-26T21:16:03.274791+0000","last_became_peered":"2024-04-26T21:16:03.274791+0000","last_unstale":"2024-04-26T21:16:56.362638+0000","last_undegraded":"2024-04-26T21:16:56.362638+0000","last_fullsized":"2024-04-26T21:16:56.362638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:15.583747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364592+0000","last_change":"2024-04-26T21:16:03.701473+0000","last_active":"2024-04-26T21:16:56.364592+0000","last_peered":"2024-04-26T21:16:56.364592+0000","last_clean":"2024-04-26T21:16:56.364592+0000","last_became_active":"2024-04-26T21:16:03.701070+0000","last_became_peered":"2024-04-26T21:16:03.701070+0000","last_unstale":"2024-04-26T21:16:56.364592+0000","last_undegraded":"2024-04-26T21:16:56.364592+0000","last_fullsized":"2024-04-26T21:16:56.364592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:25.373301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360117+0000","last_change":"2024-04-26T21:16:03.271036+0000","last_active":"2024-04-26T21:16:56.360117+0000","last_peered":"2024-04-26T21:16:56.360117+0000","last_clean":"2024-04-26T21:16:56.360117+0000","last_became_active":"2024-04-26T21:16:03.270686+0000","last_became_peered":"2024-04-26T21:16:03.270686+0000","last_unstale":"2024-04-26T21:16:56.360117+0000","last_undegraded":"2024-04-26T21:16:56.360117+0000","last_fullsized":"2024-04-26T21:16:56.360117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:21.025757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368114+0000","last_change":"2024-04-26T21:16:03.705314+0000","last_active":"2024-04-26T21:16:56.368114+0000","last_peered":"2024-04-26T21:16:56.368114+0000","last_clean":"2024-04-26T21:16:56.368114+0000","last_became_active":"2024-04-26T21:16:03.704952+0000","last_became_peered":"2024-04-26T21:16:03.704952+0000","last_unstale":"2024-04-26T21:16:56.368114+0000","last_undegraded":"2024-04-26T21:16:56.368114+0000","last_fullsized":"2024-04-26T21:16:56.368114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:57.363909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713329+0000","last_change":"2024-04-26T21:16:03.280995+0000","last_active":"2024-04-26T21:16:56.713329+0000","last_peered":"2024-04-26T21:16:56.713329+0000","last_clean":"2024-04-26T21:16:56.713329+0000","last_became_active":"2024-04-26T21:16:03.280615+0000","last_became_peered":"2024-04-26T21:16:03.280615+0000","last_unstale":"2024-04-26T21:16:56.713329+0000","last_undegraded":"2024-04-26T21:16:56.713329+0000","last_fullsized":"2024-04-26T21:16:56.713329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:32.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362097+0000","last_change":"2024-04-26T21:16:03.701218+0000","last_active":"2024-04-26T21:16:56.362097+0000","last_peered":"2024-04-26T21:16:56.362097+0000","last_clean":"2024-04-26T21:16:56.362097+0000","last_became_active":"2024-04-26T21:16:03.700895+0000","last_became_peered":"2024-04-26T21:16:03.700895+0000","last_unstale":"2024-04-26T21:16:56.362097+0000","last_undegraded":"2024-04-26T21:16:56.362097+0000","last_fullsized":"2024-04-26T21:16:56.362097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:44.104683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362857+0000","last_change":"2024-04-26T21:16:03.277235+0000","last_active":"2024-04-26T21:16:56.362857+0000","last_peered":"2024-04-26T21:16:56.362857+0000","last_clean":"2024-04-26T21:16:56.362857+0000","last_became_active":"2024-04-26T21:16:03.276901+0000","last_became_peered":"2024-04-26T21:16:03.276901+0000","last_unstale":"2024-04-26T21:16:56.362857+0000","last_undegraded":"2024-04-26T21:16:56.362857+0000","last_fullsized":"2024-04-26T21:16:56.362857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:16.297883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359135+0000","last_change":"2024-04-26T21:16:03.704318+0000","last_active":"2024-04-26T21:16:56.359135+0000","last_peered":"2024-04-26T21:16:56.359135+0000","last_clean":"2024-04-26T21:16:56.359135+0000","last_became_active":"2024-04-26T21:16:03.703907+0000","last_became_peered":"2024-04-26T21:16:03.703907+0000","last_unstale":"2024-04-26T21:16:56.359135+0000","last_undegraded":"2024-04-26T21:16:56.359135+0000","last_fullsized":"2024-04-26T21:16:56.359135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:16.431246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372105+0000","last_change":"2024-04-26T21:16:56.371135+0000","last_active":"2024-04-26T21:16:56.372105+0000","last_peered":"2024-04-26T21:16:56.372105+0000","last_clean":"2024-04-26T21:16:56.372105+0000","last_became_active":"2024-04-26T21:16:56.370666+0000","last_became_peered":"2024-04-26T21:16:56.370666+0000","last_unstale":"2024-04-26T21:16:56.372105+0000","last_undegraded":"2024-04-26T21:16:56.372105+0000","last_fullsized":"2024-04-26T21:16:56.372105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:05.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362114+0000","last_change":"2024-04-26T21:16:03.705026+0000","last_active":"2024-04-26T21:16:56.362114+0000","last_peered":"2024-04-26T21:16:56.362114+0000","last_clean":"2024-04-26T21:16:56.362114+0000","last_became_active":"2024-04-26T21:16:03.703336+0000","last_became_peered":"2024-04-26T21:16:03.703336+0000","last_unstale":"2024-04-26T21:16:56.362114+0000","last_undegraded":"2024-04-26T21:16:56.362114+0000","last_fullsized":"2024-04-26T21:16:56.362114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:11.746362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362335+0000","last_change":"2024-04-26T21:16:03.279263+0000","last_active":"2024-04-26T21:16:56.362335+0000","last_peered":"2024-04-26T21:16:56.362335+0000","last_clean":"2024-04-26T21:16:56.362335+0000","last_became_active":"2024-04-26T21:16:03.278948+0000","last_became_peered":"2024-04-26T21:16:03.278948+0000","last_unstale":"2024-04-26T21:16:56.362335+0000","last_undegraded":"2024-04-26T21:16:56.362335+0000","last_fullsized":"2024-04-26T21:16:56.362335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:39.078346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362036+0000","last_change":"2024-04-26T21:16:03.702913+0000","last_active":"2024-04-26T21:16:56.362036+0000","last_peered":"2024-04-26T21:16:56.362036+0000","last_clean":"2024-04-26T21:16:56.362036+0000","last_became_active":"2024-04-26T21:16:03.702507+0000","last_became_peered":"2024-04-26T21:16:03.702507+0000","last_unstale":"2024-04-26T21:16:56.362036+0000","last_undegraded":"2024-04-26T21:16:56.362036+0000","last_fullsized":"2024-04-26T21:16:56.362036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:51.846983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364220+0000","last_change":"2024-04-26T21:16:03.281898+0000","last_active":"2024-04-26T21:16:56.364220+0000","last_peered":"2024-04-26T21:16:56.364220+0000","last_clean":"2024-04-26T21:16:56.364220+0000","last_became_active":"2024-04-26T21:16:03.281544+0000","last_became_peered":"2024-04-26T21:16:03.281544+0000","last_unstale":"2024-04-26T21:16:56.364220+0000","last_undegraded":"2024-04-26T21:16:56.364220+0000","last_fullsized":"2024-04-26T21:16:56.364220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:39.061050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362089+0000","last_change":"2024-04-26T21:16:03.703006+0000","last_active":"2024-04-26T21:16:56.362089+0000","last_peered":"2024-04-26T21:16:56.362089+0000","last_clean":"2024-04-26T21:16:56.362089+0000","last_became_active":"2024-04-26T21:16:03.702705+0000","last_became_peered":"2024-04-26T21:16:03.702705+0000","last_unstale":"2024-04-26T21:16:56.362089+0000","last_undegraded":"2024-04-26T21:16:56.362089+0000","last_fullsized":"2024-04-26T21:16:56.362089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:11.885993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364037+0000","last_change":"2024-04-26T21:16:03.280891+0000","last_active":"2024-04-26T21:16:56.364037+0000","last_peered":"2024-04-26T21:16:56.364037+0000","last_clean":"2024-04-26T21:16:56.364037+0000","last_became_active":"2024-04-26T21:16:03.280577+0000","last_became_peered":"2024-04-26T21:16:03.280577+0000","last_unstale":"2024-04-26T21:16:56.364037+0000","last_undegraded":"2024-04-26T21:16:56.364037+0000","last_fullsized":"2024-04-26T21:16:56.364037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:28.698997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.378258+0000","last_change":"2024-04-26T21:16:56.378258+0000","last_active":"2024-04-26T21:16:56.378258+0000","last_peered":"2024-04-26T21:16:56.378258+0000","last_clean":"2024-04-26T21:16:56.378258+0000","last_became_active":"2024-04-26T21:16:56.371874+0000","last_became_peered":"2024-04-26T21:16:56.371874+0000","last_unstale":"2024-04-26T21:16:56.378258+0000","last_undegraded":"2024-04-26T21:16:56.378258+0000","last_fullsized":"2024-04-26T21:16:56.378258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:56.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360941+0000","last_change":"2024-04-26T21:16:03.276651+0000","last_active":"2024-04-26T21:16:56.360941+0000","last_peered":"2024-04-26T21:16:56.360941+0000","last_clean":"2024-04-26T21:16:56.360941+0000","last_became_active":"2024-04-26T21:16:03.276347+0000","last_became_peered":"2024-04-26T21:16:03.276347+0000","last_unstale":"2024-04-26T21:16:56.360941+0000","last_undegraded":"2024-04-26T21:16:56.360941+0000","last_fullsized":"2024-04-26T21:16:56.360941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:48.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362100+0000","last_change":"2024-04-26T21:16:03.705647+0000","last_active":"2024-04-26T21:16:56.362100+0000","last_peered":"2024-04-26T21:16:56.362100+0000","last_clean":"2024-04-26T21:16:56.362100+0000","last_became_active":"2024-04-26T21:16:03.705251+0000","last_became_peered":"2024-04-26T21:16:03.705251+0000","last_unstale":"2024-04-26T21:16:56.362100+0000","last_undegraded":"2024-04-26T21:16:56.362100+0000","last_fullsized":"2024-04-26T21:16:56.362100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:17.427221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359241+0000","last_change":"2024-04-26T21:16:03.701759+0000","last_active":"2024-04-26T21:16:56.359241+0000","last_peered":"2024-04-26T21:16:56.359241+0000","last_clean":"2024-04-26T21:16:56.359241+0000","last_became_active":"2024-04-26T21:16:03.701382+0000","last_became_peered":"2024-04-26T21:16:03.701382+0000","last_unstale":"2024-04-26T21:16:56.359241+0000","last_undegraded":"2024-04-26T21:16:56.359241+0000","last_fullsized":"2024-04-26T21:16:56.359241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.789455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372345+0000","last_change":"2024-04-26T21:16:56.372345+0000","last_active":"2024-04-26T21:16:56.372345+0000","last_peered":"2024-04-26T21:16:56.372345+0000","last_clean":"2024-04-26T21:16:56.372345+0000","last_became_active":"2024-04-26T21:16:56.371900+0000","last_became_peered":"2024-04-26T21:16:56.371900+0000","last_unstale":"2024-04-26T21:16:56.372345+0000","last_undegraded":"2024-04-26T21:16:56.372345+0000","last_fullsized":"2024-04-26T21:16:56.372345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:42.192142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361562+0000","last_change":"2024-04-26T21:16:03.702012+0000","last_active":"2024-04-26T21:16:56.361562+0000","last_peered":"2024-04-26T21:16:56.361562+0000","last_clean":"2024-04-26T21:16:56.361562+0000","last_became_active":"2024-04-26T21:16:03.701066+0000","last_became_peered":"2024-04-26T21:16:03.701066+0000","last_unstale":"2024-04-26T21:16:56.361562+0000","last_undegraded":"2024-04-26T21:16:56.361562+0000","last_fullsized":"2024-04-26T21:16:56.361562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:45.337320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367378+0000","last_change":"2024-04-26T21:16:03.702640+0000","last_active":"2024-04-26T21:16:56.367378+0000","last_peered":"2024-04-26T21:16:56.367378+0000","last_clean":"2024-04-26T21:16:56.367378+0000","last_became_active":"2024-04-26T21:16:03.702339+0000","last_became_peered":"2024-04-26T21:16:03.702339+0000","last_unstale":"2024-04-26T21:16:56.367378+0000","last_undegraded":"2024-04-26T21:16:56.367378+0000","last_fullsized":"2024-04-26T21:16:56.367378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:24.083734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361731+0000","last_change":"2024-04-26T21:16:03.708369+0000","last_active":"2024-04-26T21:16:56.361731+0000","last_peered":"2024-04-26T21:16:56.361731+0000","last_clean":"2024-04-26T21:16:56.361731+0000","last_became_active":"2024-04-26T21:16:03.708050+0000","last_became_peered":"2024-04-26T21:16:03.708050+0000","last_unstale":"2024-04-26T21:16:56.361731+0000","last_undegraded":"2024-04-26T21:16:56.361731+0000","last_fullsized":"2024-04-26T21:16:56.361731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:58.200043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359681+0000","last_change":"2024-04-26T21:16:03.700362+0000","last_active":"2024-04-26T21:16:56.359681+0000","last_peered":"2024-04-26T21:16:56.359681+0000","last_clean":"2024-04-26T21:16:56.359681+0000","last_became_active":"2024-04-26T21:16:03.700053+0000","last_became_peered":"2024-04-26T21:16:03.700053+0000","last_unstale":"2024-04-26T21:16:56.359681+0000","last_undegraded":"2024-04-26T21:16:56.359681+0000","last_fullsized":"2024-04-26T21:16:56.359681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:23.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363986+0000","last_change":"2024-04-26T21:16:03.707717+0000","last_active":"2024-04-26T21:16:56.363986+0000","last_peered":"2024-04-26T21:16:56.363986+0000","last_clean":"2024-04-26T21:16:56.363986+0000","last_became_active":"2024-04-26T21:16:03.707343+0000","last_became_peered":"2024-04-26T21:16:03.707343+0000","last_unstale":"2024-04-26T21:16:56.363986+0000","last_undegraded":"2024-04-26T21:16:56.363986+0000","last_fullsized":"2024-04-26T21:16:56.363986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:46.261231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369892+0000","last_change":"2024-04-26T21:16:56.369892+0000","last_active":"2024-04-26T21:16:56.369892+0000","last_peered":"2024-04-26T21:16:56.369892+0000","last_clean":"2024-04-26T21:16:56.369892+0000","last_became_active":"2024-04-26T21:16:56.369416+0000","last_became_peered":"2024-04-26T21:16:56.369416+0000","last_unstale":"2024-04-26T21:16:56.369892+0000","last_undegraded":"2024-04-26T21:16:56.369892+0000","last_fullsized":"2024-04-26T21:16:56.369892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:45.708969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362186+0000","last_change":"2024-04-26T21:16:03.699537+0000","last_active":"2024-04-26T21:16:56.362186+0000","last_peered":"2024-04-26T21:16:56.362186+0000","last_clean":"2024-04-26T21:16:56.362186+0000","last_became_active":"2024-04-26T21:16:03.699186+0000","last_became_peered":"2024-04-26T21:16:03.699186+0000","last_unstale":"2024-04-26T21:16:56.362186+0000","last_undegraded":"2024-04-26T21:16:56.362186+0000","last_fullsized":"2024-04-26T21:16:56.362186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:34.784929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362274+0000","last_change":"2024-04-26T21:16:03.709940+0000","last_active":"2024-04-26T21:16:56.362274+0000","last_peered":"2024-04-26T21:16:56.362274+0000","last_clean":"2024-04-26T21:16:56.362274+0000","last_became_active":"2024-04-26T21:16:03.707255+0000","last_became_peered":"2024-04-26T21:16:03.707255+0000","last_unstale":"2024-04-26T21:16:56.362274+0000","last_undegraded":"2024-04-26T21:16:56.362274+0000","last_fullsized":"2024-04-26T21:16:56.362274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:57.652194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713080+0000","last_change":"2024-04-26T21:16:03.711139+0000","last_active":"2024-04-26T21:16:56.713080+0000","last_peered":"2024-04-26T21:16:56.713080+0000","last_clean":"2024-04-26T21:16:56.713080+0000","last_became_active":"2024-04-26T21:16:03.709720+0000","last_became_peered":"2024-04-26T21:16:03.709720+0000","last_unstale":"2024-04-26T21:16:56.713080+0000","last_undegraded":"2024-04-26T21:16:56.713080+0000","last_fullsized":"2024-04-26T21:16:56.713080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:10.021914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361384+0000","last_change":"2024-04-26T21:16:03.699140+0000","last_active":"2024-04-26T21:16:56.361384+0000","last_peered":"2024-04-26T21:16:56.361384+0000","last_clean":"2024-04-26T21:16:56.361384+0000","last_became_active":"2024-04-26T21:16:03.698824+0000","last_became_peered":"2024-04-26T21:16:03.698824+0000","last_unstale":"2024-04-26T21:16:56.361384+0000","last_undegraded":"2024-04-26T21:16:56.361384+0000","last_fullsized":"2024-04-26T21:16:56.361384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:55.896278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359230+0000","last_change":"2024-04-26T21:16:03.708258+0000","last_active":"2024-04-26T21:16:56.359230+0000","last_peered":"2024-04-26T21:16:56.359230+0000","last_clean":"2024-04-26T21:16:56.359230+0000","last_became_active":"2024-04-26T21:16:03.707932+0000","last_became_peered":"2024-04-26T21:16:03.707932+0000","last_unstale":"2024-04-26T21:16:56.359230+0000","last_undegraded":"2024-04-26T21:16:56.359230+0000","last_fullsized":"2024-04-26T21:16:56.359230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:56.758307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244196,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53330,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":243,"num_read_kb":175,"num_write":77,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":160298,"data_compressed":5568,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":458793,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":1102004,"data_compressed":43255,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459492,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459397,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1049860,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":458910,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":511325,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":458883,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":511667,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":459014,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":458815,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1049460,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":52532,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52874,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":52726,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:18:07.995 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-26T21:18:07.995 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:18:08.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.130+0000 7f31af54d640 1 -- 172.21.15.129:0/600060894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 msgr2=0x7f31a8109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.130+0000 7f31af54d640 1 --2- 172.21.15.129:0/600060894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 0x7f31a8109cd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009ca0 tx=0x7f31a002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.130+0000 7f31af54d640 1 -- 172.21.15.129:0/600060894 shutdown_connections 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.130+0000 7f31af54d640 1 --2- 172.21.15.129:0/600060894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 0x7f31a8109cd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.130+0000 7f31af54d640 1 --2- 172.21.15.129:0/600060894 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8102a80 0x7f31a8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.130+0000 7f31af54d640 1 --2- 172.21.15.129:0/600060894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8101890 0x7f31a8101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.130+0000 7f31af54d640 1 -- 172.21.15.129:0/600060894 >> 172.21.15.129:0/600060894 conn(0x7f31a80fce20 msgr2=0x7f31a80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.130+0000 7f31af54d640 1 -- 172.21.15.129:0/600060894 shutdown_connections 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.130+0000 7f31af54d640 1 -- 172.21.15.129:0/600060894 wait complete. 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31af54d640 1 Processor -- start 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31af54d640 1 -- start start 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31af54d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8101890 0x7f31a8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31af54d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8102a80 0x7f31a8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31af54d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 0x7f31a819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31af54d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31a810d3a0 con 0x7f31a8103440 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31af54d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31a810d220 con 0x7f31a8102a80 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31af54d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f31a810d520 con 0x7f31a8101890 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31adaea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 0x7f31a819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31adaea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 0x7f31a819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59902/0 (socket says 172.21.15.129:59902) 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31adaea640 1 -- 172.21.15.129:0/2827624330 learned_addr learned my addr 172.21.15.129:0/2827624330 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31ad2e9640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8101890 0x7f31a8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31acae8640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8102a80 0x7f31a8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31adaea640 1 -- 172.21.15.129:0/2827624330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8101890 msgr2=0x7f31a8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31adaea640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8101890 0x7f31a8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31adaea640 1 -- 172.21.15.129:0/2827624330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8102a80 msgr2=0x7f31a8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31adaea640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8102a80 0x7f31a8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31adaea640 1 -- 172.21.15.129:0/2827624330 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a819f690 con 0x7f31a8103440 2024-04-26T21:18:08.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.131+0000 7f31adaea640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 0x7f31a819f020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f31a00097b0 tx=0x7f31a0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:08.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.132+0000 7f319e7fc640 1 -- 172.21.15.129:0/2827624330 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0004240 con 0x7f31a8103440 2024-04-26T21:18:08.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.132+0000 7f319e7fc640 1 -- 172.21.15.129:0/2827624330 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a00317c0 con 0x7f31a8103440 2024-04-26T21:18:08.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.132+0000 7f319e7fc640 1 -- 172.21.15.129:0/2827624330 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a002fd80 con 0x7f31a8103440 2024-04-26T21:18:08.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.132+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a819f920 con 0x7f31a8103440 2024-04-26T21:18:08.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.132+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a819fe70 con 0x7f31a8103440 2024-04-26T21:18:08.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.133+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a8108c50 con 0x7f31a8103440 2024-04-26T21:18:08.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.134+0000 7f319e7fc640 1 -- 172.21.15.129:0/2827624330 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f31a00043e0 con 0x7f31a8103440 2024-04-26T21:18:08.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.134+0000 7f319e7fc640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.134+0000 7f31ad2e9640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.135+0000 7f319e7fc640 1 -- 172.21.15.129:0/2827624330 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f31a00f1d20 con 0x7f31a8103440 2024-04-26T21:18:08.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.135+0000 7f31ad2e9640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f31a81028e0 tx=0x7f31a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:08.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.139+0000 7f319e7fc640 1 -- 172.21.15.129:0/2827624330 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f31a00be6a0 con 0x7f31a8103440 2024-04-26T21:18:08.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.312+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 --> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31a8062d70 con 0x7f31940a3c40 2024-04-26T21:18:08.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.319+0000 7f319e7fc640 1 -- 172.21.15.129:0/2827624330 <== mgr.14152 v2:172.21.15.129:6800/3856000307 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7f31a8062d70 con 0x7f31940a3c40 2024-04-26T21:18:08.323 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:08.324 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-26T21:18:08.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 msgr2=0x7f31940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f31a81028e0 tx=0x7f31a400b390 comp rx=0 tx=0).stop 2024-04-26T21:18:08.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 msgr2=0x7f31a819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 0x7f31a819f020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f31a00097b0 tx=0x7f31a0003c20 comp rx=0 tx=0).stop 2024-04-26T21:18:08.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 shutdown_connections 2024-04-26T21:18:08.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31940a3c40 0x7f31940a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a8102a80 0x7f31a8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31a8103440 0x7f31a819f020 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 --2- 172.21.15.129:0/2827624330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31a8101890 0x7f31a8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 >> 172.21.15.129:0/2827624330 conn(0x7f31a80fce20 msgr2=0x7f31a8105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:08.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.322+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 shutdown_connections 2024-04-26T21:18:08.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.323+0000 7f31af54d640 1 -- 172.21.15.129:0/2827624330 wait complete. 2024-04-26T21:18:08.335 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-04-26T21:18:07.308293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643932,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":369,"num_read_kb":284,"num_write":260,"num_write_kb":3015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360696,"kb_used_data":11484,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581128,"statfs":{"total":1151940427776,"available":1151571075072,"internally_reserved":0,"allocated":11759616,"data_stored":7437620,"data_compressed":182064,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002752"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359578+0000","last_change":"2024-04-26T21:16:03.700581+0000","last_active":"2024-04-26T21:16:56.359578+0000","last_peered":"2024-04-26T21:16:56.359578+0000","last_clean":"2024-04-26T21:16:56.359578+0000","last_became_active":"2024-04-26T21:16:03.700280+0000","last_became_peered":"2024-04-26T21:16:03.700280+0000","last_unstale":"2024-04-26T21:16:56.359578+0000","last_undegraded":"2024-04-26T21:16:56.359578+0000","last_fullsized":"2024-04-26T21:16:56.359578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:01.329784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368025+0000","last_change":"2024-04-26T21:16:03.702058+0000","last_active":"2024-04-26T21:16:56.368025+0000","last_peered":"2024-04-26T21:16:56.368025+0000","last_clean":"2024-04-26T21:16:56.368025+0000","last_became_active":"2024-04-26T21:16:03.701728+0000","last_became_peered":"2024-04-26T21:16:03.701728+0000","last_unstale":"2024-04-26T21:16:56.368025+0000","last_undegraded":"2024-04-26T21:16:56.368025+0000","last_fullsized":"2024-04-26T21:16:56.368025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:12.718225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367712+0000","last_change":"2024-04-26T21:16:03.710461+0000","last_active":"2024-04-26T21:16:56.367712+0000","last_peered":"2024-04-26T21:16:56.367712+0000","last_clean":"2024-04-26T21:16:56.367712+0000","last_became_active":"2024-04-26T21:16:03.710130+0000","last_became_peered":"2024-04-26T21:16:03.710130+0000","last_unstale":"2024-04-26T21:16:56.367712+0000","last_undegraded":"2024-04-26T21:16:56.367712+0000","last_fullsized":"2024-04-26T21:16:56.367712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:21.294677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360217+0000","last_change":"2024-04-26T21:16:03.706836+0000","last_active":"2024-04-26T21:16:56.360217+0000","last_peered":"2024-04-26T21:16:56.360217+0000","last_clean":"2024-04-26T21:16:56.360217+0000","last_became_active":"2024-04-26T21:16:03.706515+0000","last_became_peered":"2024-04-26T21:16:03.706515+0000","last_unstale":"2024-04-26T21:16:56.360217+0000","last_undegraded":"2024-04-26T21:16:56.360217+0000","last_fullsized":"2024-04-26T21:16:56.360217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:56.198337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362204+0000","last_change":"2024-04-26T21:16:03.704865+0000","last_active":"2024-04-26T21:16:56.362204+0000","last_peered":"2024-04-26T21:16:56.362204+0000","last_clean":"2024-04-26T21:16:56.362204+0000","last_became_active":"2024-04-26T21:16:03.704560+0000","last_became_peered":"2024-04-26T21:16:03.704560+0000","last_unstale":"2024-04-26T21:16:56.362204+0000","last_undegraded":"2024-04-26T21:16:56.362204+0000","last_fullsized":"2024-04-26T21:16:56.362204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:51:03.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712989+0000","last_change":"2024-04-26T21:16:03.700265+0000","last_active":"2024-04-26T21:16:56.712989+0000","last_peered":"2024-04-26T21:16:56.712989+0000","last_clean":"2024-04-26T21:16:56.712989+0000","last_became_active":"2024-04-26T21:16:03.699878+0000","last_became_peered":"2024-04-26T21:16:03.699878+0000","last_unstale":"2024-04-26T21:16:56.712989+0000","last_undegraded":"2024-04-26T21:16:56.712989+0000","last_fullsized":"2024-04-26T21:16:56.712989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:12.907254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361631+0000","last_change":"2024-04-26T21:16:03.697569+0000","last_active":"2024-04-26T21:16:56.361631+0000","last_peered":"2024-04-26T21:16:56.361631+0000","last_clean":"2024-04-26T21:16:56.361631+0000","last_became_active":"2024-04-26T21:16:03.697262+0000","last_became_peered":"2024-04-26T21:16:03.697262+0000","last_unstale":"2024-04-26T21:16:56.361631+0000","last_undegraded":"2024-04-26T21:16:56.361631+0000","last_fullsized":"2024-04-26T21:16:56.361631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:28.645750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363218+0000","last_change":"2024-04-26T21:16:03.705788+0000","last_active":"2024-04-26T21:16:56.363218+0000","last_peered":"2024-04-26T21:16:56.363218+0000","last_clean":"2024-04-26T21:16:56.363218+0000","last_became_active":"2024-04-26T21:16:03.705479+0000","last_became_peered":"2024-04-26T21:16:03.705479+0000","last_unstale":"2024-04-26T21:16:56.363218+0000","last_undegraded":"2024-04-26T21:16:56.363218+0000","last_fullsized":"2024-04-26T21:16:56.363218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:27.306777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713655+0000","last_change":"2024-04-26T21:16:03.709716+0000","last_active":"2024-04-26T21:16:56.713655+0000","last_peered":"2024-04-26T21:16:56.713655+0000","last_clean":"2024-04-26T21:16:56.713655+0000","last_became_active":"2024-04-26T21:16:03.709219+0000","last_became_peered":"2024-04-26T21:16:03.709219+0000","last_unstale":"2024-04-26T21:16:56.713655+0000","last_undegraded":"2024-04-26T21:16:56.713655+0000","last_fullsized":"2024-04-26T21:16:56.713655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:18.936075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713179+0000","last_change":"2024-04-26T21:16:03.694745+0000","last_active":"2024-04-26T21:16:56.713179+0000","last_peered":"2024-04-26T21:16:56.713179+0000","last_clean":"2024-04-26T21:16:56.713179+0000","last_became_active":"2024-04-26T21:16:03.694044+0000","last_became_peered":"2024-04-26T21:16:03.694044+0000","last_unstale":"2024-04-26T21:16:56.713179+0000","last_undegraded":"2024-04-26T21:16:56.713179+0000","last_fullsized":"2024-04-26T21:16:56.713179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:13.060467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362033+0000","last_change":"2024-04-26T21:16:03.706434+0000","last_active":"2024-04-26T21:16:56.362033+0000","last_peered":"2024-04-26T21:16:56.362033+0000","last_clean":"2024-04-26T21:16:56.362033+0000","last_became_active":"2024-04-26T21:16:03.706093+0000","last_became_peered":"2024-04-26T21:16:03.706093+0000","last_unstale":"2024-04-26T21:16:56.362033+0000","last_undegraded":"2024-04-26T21:16:56.362033+0000","last_fullsized":"2024-04-26T21:16:56.362033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:04.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360084+0000","last_change":"2024-04-26T21:16:03.707672+0000","last_active":"2024-04-26T21:16:56.360084+0000","last_peered":"2024-04-26T21:16:56.360084+0000","last_clean":"2024-04-26T21:16:56.360084+0000","last_became_active":"2024-04-26T21:16:03.707344+0000","last_became_peered":"2024-04-26T21:16:03.707344+0000","last_unstale":"2024-04-26T21:16:56.360084+0000","last_undegraded":"2024-04-26T21:16:56.360084+0000","last_fullsized":"2024-04-26T21:16:56.360084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:07.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361281+0000","last_change":"2024-04-26T21:16:03.699816+0000","last_active":"2024-04-26T21:16:56.361281+0000","last_peered":"2024-04-26T21:16:56.361281+0000","last_clean":"2024-04-26T21:16:56.361281+0000","last_became_active":"2024-04-26T21:16:03.699514+0000","last_became_peered":"2024-04-26T21:16:03.699514+0000","last_unstale":"2024-04-26T21:16:56.361281+0000","last_undegraded":"2024-04-26T21:16:56.361281+0000","last_fullsized":"2024-04-26T21:16:56.361281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:59.230438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367152+0000","last_change":"2024-04-26T21:16:03.709627+0000","last_active":"2024-04-26T21:16:56.367152+0000","last_peered":"2024-04-26T21:16:56.367152+0000","last_clean":"2024-04-26T21:16:56.367152+0000","last_became_active":"2024-04-26T21:16:03.709238+0000","last_became_peered":"2024-04-26T21:16:03.709238+0000","last_unstale":"2024-04-26T21:16:56.367152+0000","last_undegraded":"2024-04-26T21:16:56.367152+0000","last_fullsized":"2024-04-26T21:16:56.367152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:55.303059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.366111+0000","last_change":"2024-04-26T21:16:03.702275+0000","last_active":"2024-04-26T21:16:56.366111+0000","last_peered":"2024-04-26T21:16:56.366111+0000","last_clean":"2024-04-26T21:16:56.366111+0000","last_became_active":"2024-04-26T21:16:03.701966+0000","last_became_peered":"2024-04-26T21:16:03.701966+0000","last_unstale":"2024-04-26T21:16:56.366111+0000","last_undegraded":"2024-04-26T21:16:56.366111+0000","last_fullsized":"2024-04-26T21:16:56.366111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:02.299759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713281+0000","last_change":"2024-04-26T21:16:03.711979+0000","last_active":"2024-04-26T21:16:56.713281+0000","last_peered":"2024-04-26T21:16:56.713281+0000","last_clean":"2024-04-26T21:16:56.713281+0000","last_became_active":"2024-04-26T21:16:03.711637+0000","last_became_peered":"2024-04-26T21:16:03.711637+0000","last_unstale":"2024-04-26T21:16:56.713281+0000","last_undegraded":"2024-04-26T21:16:56.713281+0000","last_fullsized":"2024-04-26T21:16:56.713281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:27.817993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359639+0000","last_change":"2024-04-26T21:16:03.701303+0000","last_active":"2024-04-26T21:16:56.359639+0000","last_peered":"2024-04-26T21:16:56.359639+0000","last_clean":"2024-04-26T21:16:56.359639+0000","last_became_active":"2024-04-26T21:16:03.701001+0000","last_became_peered":"2024-04-26T21:16:03.701001+0000","last_unstale":"2024-04-26T21:16:56.359639+0000","last_undegraded":"2024-04-26T21:16:56.359639+0000","last_fullsized":"2024-04-26T21:16:56.359639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:08.349915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362455+0000","last_change":"2024-04-26T21:16:03.696775+0000","last_active":"2024-04-26T21:16:56.362455+0000","last_peered":"2024-04-26T21:16:56.362455+0000","last_clean":"2024-04-26T21:16:56.362455+0000","last_became_active":"2024-04-26T21:16:03.696433+0000","last_became_peered":"2024-04-26T21:16:03.696433+0000","last_unstale":"2024-04-26T21:16:56.362455+0000","last_undegraded":"2024-04-26T21:16:56.362455+0000","last_fullsized":"2024-04-26T21:16:56.362455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:50.029169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361906+0000","last_change":"2024-04-26T21:16:03.698188+0000","last_active":"2024-04-26T21:16:56.361906+0000","last_peered":"2024-04-26T21:16:56.361906+0000","last_clean":"2024-04-26T21:16:56.361906+0000","last_became_active":"2024-04-26T21:16:03.697882+0000","last_became_peered":"2024-04-26T21:16:03.697882+0000","last_unstale":"2024-04-26T21:16:56.361906+0000","last_undegraded":"2024-04-26T21:16:56.361906+0000","last_fullsized":"2024-04-26T21:16:56.361906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:04.737619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362709+0000","last_change":"2024-04-26T21:16:03.275296+0000","last_active":"2024-04-26T21:16:56.362709+0000","last_peered":"2024-04-26T21:16:56.362709+0000","last_clean":"2024-04-26T21:16:56.362709+0000","last_became_active":"2024-04-26T21:16:03.274733+0000","last_became_peered":"2024-04-26T21:16:03.274733+0000","last_unstale":"2024-04-26T21:16:56.362709+0000","last_undegraded":"2024-04-26T21:16:56.362709+0000","last_fullsized":"2024-04-26T21:16:56.362709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:16.512934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363299+0000","last_change":"2024-04-26T21:16:03.705823+0000","last_active":"2024-04-26T21:16:56.363299+0000","last_peered":"2024-04-26T21:16:56.363299+0000","last_clean":"2024-04-26T21:16:56.363299+0000","last_became_active":"2024-04-26T21:16:03.705512+0000","last_became_peered":"2024-04-26T21:16:03.705512+0000","last_unstale":"2024-04-26T21:16:56.363299+0000","last_undegraded":"2024-04-26T21:16:56.363299+0000","last_fullsized":"2024-04-26T21:16:56.363299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:17.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364347+0000","last_change":"2024-04-26T21:16:03.285008+0000","last_active":"2024-04-26T21:16:56.364347+0000","last_peered":"2024-04-26T21:16:56.364347+0000","last_clean":"2024-04-26T21:16:56.364347+0000","last_became_active":"2024-04-26T21:16:03.284679+0000","last_became_peered":"2024-04-26T21:16:03.284679+0000","last_unstale":"2024-04-26T21:16:56.364347+0000","last_undegraded":"2024-04-26T21:16:56.364347+0000","last_fullsized":"2024-04-26T21:16:56.364347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:35.404086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368858+0000","last_change":"2024-04-26T21:16:03.710571+0000","last_active":"2024-04-26T21:16:56.368858+0000","last_peered":"2024-04-26T21:16:56.368858+0000","last_clean":"2024-04-26T21:16:56.368858+0000","last_became_active":"2024-04-26T21:16:03.710237+0000","last_became_peered":"2024-04-26T21:16:03.710237+0000","last_unstale":"2024-04-26T21:16:56.368858+0000","last_undegraded":"2024-04-26T21:16:56.368858+0000","last_fullsized":"2024-04-26T21:16:56.368858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:53.188554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369209+0000","last_change":"2024-04-26T21:16:56.368746+0000","last_active":"2024-04-26T21:16:56.369209+0000","last_peered":"2024-04-26T21:16:56.369209+0000","last_clean":"2024-04-26T21:16:56.369209+0000","last_became_active":"2024-04-26T21:16:56.365062+0000","last_became_peered":"2024-04-26T21:16:56.365062+0000","last_unstale":"2024-04-26T21:16:56.369209+0000","last_undegraded":"2024-04-26T21:16:56.369209+0000","last_fullsized":"2024-04-26T21:16:56.369209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:36.165146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713534+0000","last_change":"2024-04-26T21:16:03.711355+0000","last_active":"2024-04-26T21:16:56.713534+0000","last_peered":"2024-04-26T21:16:56.713534+0000","last_clean":"2024-04-26T21:16:56.713534+0000","last_became_active":"2024-04-26T21:16:03.710885+0000","last_became_peered":"2024-04-26T21:16:03.710885+0000","last_unstale":"2024-04-26T21:16:56.713534+0000","last_undegraded":"2024-04-26T21:16:56.713534+0000","last_fullsized":"2024-04-26T21:16:56.713534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:10.515590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362768+0000","last_change":"2024-04-26T21:16:03.278484+0000","last_active":"2024-04-26T21:16:56.362768+0000","last_peered":"2024-04-26T21:16:56.362768+0000","last_clean":"2024-04-26T21:16:56.362768+0000","last_became_active":"2024-04-26T21:16:03.277402+0000","last_became_peered":"2024-04-26T21:16:03.277402+0000","last_unstale":"2024-04-26T21:16:56.362768+0000","last_undegraded":"2024-04-26T21:16:56.362768+0000","last_fullsized":"2024-04-26T21:16:56.362768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:21.807942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714671+0000","last_change":"2024-04-26T21:16:56.714671+0000","last_active":"2024-04-26T21:16:56.714671+0000","last_peered":"2024-04-26T21:16:56.714671+0000","last_clean":"2024-04-26T21:16:56.714671+0000","last_became_active":"2024-04-26T21:16:56.714209+0000","last_became_peered":"2024-04-26T21:16:56.714209+0000","last_unstale":"2024-04-26T21:16:56.714671+0000","last_undegraded":"2024-04-26T21:16:56.714671+0000","last_fullsized":"2024-04-26T21:16:56.714671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:44.790915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363442+0000","last_change":"2024-04-26T21:16:03.277219+0000","last_active":"2024-04-26T21:16:56.363442+0000","last_peered":"2024-04-26T21:16:56.363442+0000","last_clean":"2024-04-26T21:16:56.363442+0000","last_became_active":"2024-04-26T21:16:03.276885+0000","last_became_peered":"2024-04-26T21:16:03.276885+0000","last_unstale":"2024-04-26T21:16:56.363442+0000","last_undegraded":"2024-04-26T21:16:56.363442+0000","last_fullsized":"2024-04-26T21:16:56.363442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:45.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362963+0000","last_change":"2024-04-26T21:16:03.703453+0000","last_active":"2024-04-26T21:16:56.362963+0000","last_peered":"2024-04-26T21:16:56.362963+0000","last_clean":"2024-04-26T21:16:56.362963+0000","last_became_active":"2024-04-26T21:16:03.703161+0000","last_became_peered":"2024-04-26T21:16:03.703161+0000","last_unstale":"2024-04-26T21:16:56.362963+0000","last_undegraded":"2024-04-26T21:16:56.362963+0000","last_fullsized":"2024-04-26T21:16:56.362963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.999243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712837+0000","last_change":"2024-04-26T21:16:03.274815+0000","last_active":"2024-04-26T21:16:56.712837+0000","last_peered":"2024-04-26T21:16:56.712837+0000","last_clean":"2024-04-26T21:16:56.712837+0000","last_became_active":"2024-04-26T21:16:03.274509+0000","last_became_peered":"2024-04-26T21:16:03.274509+0000","last_unstale":"2024-04-26T21:16:56.712837+0000","last_undegraded":"2024-04-26T21:16:56.712837+0000","last_fullsized":"2024-04-26T21:16:56.712837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:27.212714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.371100+0000","last_change":"2024-04-26T21:16:56.371100+0000","last_active":"2024-04-26T21:16:56.371100+0000","last_peered":"2024-04-26T21:16:56.371100+0000","last_clean":"2024-04-26T21:16:56.371100+0000","last_became_active":"2024-04-26T21:16:56.370711+0000","last_became_peered":"2024-04-26T21:16:56.370711+0000","last_unstale":"2024-04-26T21:16:56.371100+0000","last_undegraded":"2024-04-26T21:16:56.371100+0000","last_fullsized":"2024-04-26T21:16:56.371100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:00.143815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360698+0000","last_change":"2024-04-26T21:16:03.270417+0000","last_active":"2024-04-26T21:16:56.360698+0000","last_peered":"2024-04-26T21:16:56.360698+0000","last_clean":"2024-04-26T21:16:56.360698+0000","last_became_active":"2024-04-26T21:16:03.270066+0000","last_became_peered":"2024-04-26T21:16:03.270066+0000","last_unstale":"2024-04-26T21:16:56.360698+0000","last_undegraded":"2024-04-26T21:16:56.360698+0000","last_fullsized":"2024-04-26T21:16:56.360698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:46.794698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363117+0000","last_change":"2024-04-26T21:16:03.700221+0000","last_active":"2024-04-26T21:16:56.363117+0000","last_peered":"2024-04-26T21:16:56.363117+0000","last_clean":"2024-04-26T21:16:56.363117+0000","last_became_active":"2024-04-26T21:16:03.699798+0000","last_became_peered":"2024-04-26T21:16:03.699798+0000","last_unstale":"2024-04-26T21:16:56.363117+0000","last_undegraded":"2024-04-26T21:16:56.363117+0000","last_fullsized":"2024-04-26T21:16:56.363117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:52.083700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362074+0000","last_change":"2024-04-26T21:16:03.282355+0000","last_active":"2024-04-26T21:16:56.362074+0000","last_peered":"2024-04-26T21:16:56.362074+0000","last_clean":"2024-04-26T21:16:56.362074+0000","last_became_active":"2024-04-26T21:16:03.281169+0000","last_became_peered":"2024-04-26T21:16:03.281169+0000","last_unstale":"2024-04-26T21:16:56.362074+0000","last_undegraded":"2024-04-26T21:16:56.362074+0000","last_fullsized":"2024-04-26T21:16:56.362074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:55.522530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.714327+0000","last_change":"2024-04-26T21:16:56.714327+0000","last_active":"2024-04-26T21:16:56.714327+0000","last_peered":"2024-04-26T21:16:56.714327+0000","last_clean":"2024-04-26T21:16:56.714327+0000","last_became_active":"2024-04-26T21:16:56.713967+0000","last_became_peered":"2024-04-26T21:16:56.713967+0000","last_unstale":"2024-04-26T21:16:56.714327+0000","last_undegraded":"2024-04-26T21:16:56.714327+0000","last_fullsized":"2024-04-26T21:16:56.714327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:32.475044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713294+0000","last_change":"2024-04-26T21:16:03.284948+0000","last_active":"2024-04-26T21:16:56.713294+0000","last_peered":"2024-04-26T21:16:56.713294+0000","last_clean":"2024-04-26T21:16:56.713294+0000","last_became_active":"2024-04-26T21:16:03.284570+0000","last_became_peered":"2024-04-26T21:16:03.284570+0000","last_unstale":"2024-04-26T21:16:56.713294+0000","last_undegraded":"2024-04-26T21:16:56.713294+0000","last_fullsized":"2024-04-26T21:16:56.713294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:22.883834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713580+0000","last_change":"2024-04-26T21:16:03.710359+0000","last_active":"2024-04-26T21:16:56.713580+0000","last_peered":"2024-04-26T21:16:56.713580+0000","last_clean":"2024-04-26T21:16:56.713580+0000","last_became_active":"2024-04-26T21:16:03.710053+0000","last_became_peered":"2024-04-26T21:16:03.710053+0000","last_unstale":"2024-04-26T21:16:56.713580+0000","last_undegraded":"2024-04-26T21:16:56.713580+0000","last_fullsized":"2024-04-26T21:16:56.713580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:45.981083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361874+0000","last_change":"2024-04-26T21:16:03.278787+0000","last_active":"2024-04-26T21:16:56.361874+0000","last_peered":"2024-04-26T21:16:56.361874+0000","last_clean":"2024-04-26T21:16:56.361874+0000","last_became_active":"2024-04-26T21:16:03.278483+0000","last_became_peered":"2024-04-26T21:16:03.278483+0000","last_unstale":"2024-04-26T21:16:56.361874+0000","last_undegraded":"2024-04-26T21:16:56.361874+0000","last_fullsized":"2024-04-26T21:16:56.361874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:40.303372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372769+0000","last_change":"2024-04-26T21:16:56.372769+0000","last_active":"2024-04-26T21:16:56.372769+0000","last_peered":"2024-04-26T21:16:56.372769+0000","last_clean":"2024-04-26T21:16:56.372769+0000","last_became_active":"2024-04-26T21:16:56.372393+0000","last_became_peered":"2024-04-26T21:16:56.372393+0000","last_unstale":"2024-04-26T21:16:56.372769+0000","last_undegraded":"2024-04-26T21:16:56.372769+0000","last_fullsized":"2024-04-26T21:16:56.372769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:29.818635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364174+0000","last_change":"2024-04-26T21:16:03.277387+0000","last_active":"2024-04-26T21:16:56.364174+0000","last_peered":"2024-04-26T21:16:56.364174+0000","last_clean":"2024-04-26T21:16:56.364174+0000","last_became_active":"2024-04-26T21:16:03.274765+0000","last_became_peered":"2024-04-26T21:16:03.274765+0000","last_unstale":"2024-04-26T21:16:56.364174+0000","last_undegraded":"2024-04-26T21:16:56.364174+0000","last_fullsized":"2024-04-26T21:16:56.364174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:51.081883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713473+0000","last_change":"2024-04-26T21:16:03.710977+0000","last_active":"2024-04-26T21:16:56.713473+0000","last_peered":"2024-04-26T21:16:56.713473+0000","last_clean":"2024-04-26T21:16:56.713473+0000","last_became_active":"2024-04-26T21:16:03.710284+0000","last_became_peered":"2024-04-26T21:16:03.710284+0000","last_unstale":"2024-04-26T21:16:56.713473+0000","last_undegraded":"2024-04-26T21:16:56.713473+0000","last_fullsized":"2024-04-26T21:16:56.713473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:46.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'25","reported_seq":281,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:18:01.869515+0000","last_change":"2024-04-26T21:16:03.276865+0000","last_active":"2024-04-26T21:18:01.869515+0000","last_peered":"2024-04-26T21:18:01.869515+0000","last_clean":"2024-04-26T21:18:01.869515+0000","last_became_active":"2024-04-26T21:16:03.276564+0000","last_became_peered":"2024-04-26T21:16:03.276564+0000","last_unstale":"2024-04-26T21:18:01.869515+0000","last_undegraded":"2024-04-26T21:18:01.869515+0000","last_fullsized":"2024-04-26T21:18:01.869515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:33.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52532,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":237,"num_read_kb":169,"num_write":25,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362088+0000","last_change":"2024-04-26T21:16:03.700441+0000","last_active":"2024-04-26T21:16:56.362088+0000","last_peered":"2024-04-26T21:16:56.362088+0000","last_clean":"2024-04-26T21:16:56.362088+0000","last_became_active":"2024-04-26T21:16:03.700142+0000","last_became_peered":"2024-04-26T21:16:03.700142+0000","last_unstale":"2024-04-26T21:16:56.362088+0000","last_undegraded":"2024-04-26T21:16:56.362088+0000","last_fullsized":"2024-04-26T21:16:56.362088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:35.166603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361872+0000","last_change":"2024-04-26T21:16:03.280144+0000","last_active":"2024-04-26T21:16:56.361872+0000","last_peered":"2024-04-26T21:16:56.361872+0000","last_clean":"2024-04-26T21:16:56.361872+0000","last_became_active":"2024-04-26T21:16:03.279844+0000","last_became_peered":"2024-04-26T21:16:03.279844+0000","last_unstale":"2024-04-26T21:16:56.361872+0000","last_undegraded":"2024-04-26T21:16:56.361872+0000","last_fullsized":"2024-04-26T21:16:56.361872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:00.704181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361493+0000","last_change":"2024-04-26T21:16:03.701732+0000","last_active":"2024-04-26T21:16:56.361493+0000","last_peered":"2024-04-26T21:16:56.361493+0000","last_clean":"2024-04-26T21:16:56.361493+0000","last_became_active":"2024-04-26T21:16:03.701424+0000","last_became_peered":"2024-04-26T21:16:03.701424+0000","last_unstale":"2024-04-26T21:16:56.361493+0000","last_undegraded":"2024-04-26T21:16:56.361493+0000","last_fullsized":"2024-04-26T21:16:56.361493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:02.293086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363077+0000","last_change":"2024-04-26T21:16:03.276816+0000","last_active":"2024-04-26T21:16:56.363077+0000","last_peered":"2024-04-26T21:16:56.363077+0000","last_clean":"2024-04-26T21:16:56.363077+0000","last_became_active":"2024-04-26T21:16:03.276471+0000","last_became_peered":"2024-04-26T21:16:03.276471+0000","last_unstale":"2024-04-26T21:16:56.363077+0000","last_undegraded":"2024-04-26T21:16:56.363077+0000","last_fullsized":"2024-04-26T21:16:56.363077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:44.630638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361866+0000","last_change":"2024-04-26T21:16:03.701002+0000","last_active":"2024-04-26T21:16:56.361866+0000","last_peered":"2024-04-26T21:16:56.361866+0000","last_clean":"2024-04-26T21:16:56.361866+0000","last_became_active":"2024-04-26T21:16:03.700649+0000","last_became_peered":"2024-04-26T21:16:03.700649+0000","last_unstale":"2024-04-26T21:16:56.361866+0000","last_undegraded":"2024-04-26T21:16:56.361866+0000","last_fullsized":"2024-04-26T21:16:56.361866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:45.571735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359679+0000","last_change":"2024-04-26T21:16:03.273727+0000","last_active":"2024-04-26T21:16:56.359679+0000","last_peered":"2024-04-26T21:16:56.359679+0000","last_clean":"2024-04-26T21:16:56.359679+0000","last_became_active":"2024-04-26T21:16:03.273287+0000","last_became_peered":"2024-04-26T21:16:03.273287+0000","last_unstale":"2024-04-26T21:16:56.359679+0000","last_undegraded":"2024-04-26T21:16:56.359679+0000","last_fullsized":"2024-04-26T21:16:56.359679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:48.713646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361559+0000","last_change":"2024-04-26T21:16:03.696760+0000","last_active":"2024-04-26T21:16:56.361559+0000","last_peered":"2024-04-26T21:16:56.361559+0000","last_clean":"2024-04-26T21:16:56.361559+0000","last_became_active":"2024-04-26T21:16:03.696446+0000","last_became_peered":"2024-04-26T21:16:03.696446+0000","last_unstale":"2024-04-26T21:16:56.361559+0000","last_undegraded":"2024-04-26T21:16:56.361559+0000","last_fullsized":"2024-04-26T21:16:56.361559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:30.710360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361962+0000","last_change":"2024-04-26T21:16:03.282643+0000","last_active":"2024-04-26T21:16:56.361962+0000","last_peered":"2024-04-26T21:16:56.361962+0000","last_clean":"2024-04-26T21:16:56.361962+0000","last_became_active":"2024-04-26T21:16:03.282315+0000","last_became_peered":"2024-04-26T21:16:03.282315+0000","last_unstale":"2024-04-26T21:16:56.361962+0000","last_undegraded":"2024-04-26T21:16:56.361962+0000","last_fullsized":"2024-04-26T21:16:56.361962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:13.358002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362813+0000","last_change":"2024-04-26T21:16:03.711992+0000","last_active":"2024-04-26T21:16:56.362813+0000","last_peered":"2024-04-26T21:16:56.362813+0000","last_clean":"2024-04-26T21:16:56.362813+0000","last_became_active":"2024-04-26T21:16:03.711755+0000","last_became_peered":"2024-04-26T21:16:03.711755+0000","last_unstale":"2024-04-26T21:16:56.362813+0000","last_undegraded":"2024-04-26T21:16:56.362813+0000","last_fullsized":"2024-04-26T21:16:56.362813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:01.321530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362836+0000","last_change":"2024-04-26T21:16:03.274722+0000","last_active":"2024-04-26T21:16:56.362836+0000","last_peered":"2024-04-26T21:16:56.362836+0000","last_clean":"2024-04-26T21:16:56.362836+0000","last_became_active":"2024-04-26T21:16:03.274399+0000","last_became_peered":"2024-04-26T21:16:03.274399+0000","last_unstale":"2024-04-26T21:16:56.362836+0000","last_undegraded":"2024-04-26T21:16:56.362836+0000","last_fullsized":"2024-04-26T21:16:56.362836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:35.247456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359442+0000","last_change":"2024-04-26T21:16:03.698344+0000","last_active":"2024-04-26T21:16:56.359442+0000","last_peered":"2024-04-26T21:16:56.359442+0000","last_clean":"2024-04-26T21:16:56.359442+0000","last_became_active":"2024-04-26T21:16:03.698037+0000","last_became_peered":"2024-04-26T21:16:03.698037+0000","last_unstale":"2024-04-26T21:16:56.359442+0000","last_undegraded":"2024-04-26T21:16:56.359442+0000","last_fullsized":"2024-04-26T21:16:56.359442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.573421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362176+0000","last_change":"2024-04-26T21:16:03.276459+0000","last_active":"2024-04-26T21:16:56.362176+0000","last_peered":"2024-04-26T21:16:56.362176+0000","last_clean":"2024-04-26T21:16:56.362176+0000","last_became_active":"2024-04-26T21:16:03.276108+0000","last_became_peered":"2024-04-26T21:16:03.276108+0000","last_unstale":"2024-04-26T21:16:56.362176+0000","last_undegraded":"2024-04-26T21:16:56.362176+0000","last_fullsized":"2024-04-26T21:16:56.362176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.364925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.712741+0000","last_change":"2024-04-26T21:16:03.696030+0000","last_active":"2024-04-26T21:16:56.712741+0000","last_peered":"2024-04-26T21:16:56.712741+0000","last_clean":"2024-04-26T21:16:56.712741+0000","last_became_active":"2024-04-26T21:16:03.695717+0000","last_became_peered":"2024-04-26T21:16:03.695717+0000","last_unstale":"2024-04-26T21:16:56.712741+0000","last_undegraded":"2024-04-26T21:16:56.712741+0000","last_fullsized":"2024-04-26T21:16:56.712741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:55.702216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361784+0000","last_change":"2024-04-26T21:16:03.269188+0000","last_active":"2024-04-26T21:16:56.361784+0000","last_peered":"2024-04-26T21:16:56.361784+0000","last_clean":"2024-04-26T21:16:56.361784+0000","last_became_active":"2024-04-26T21:16:03.268850+0000","last_became_peered":"2024-04-26T21:16:03.268850+0000","last_unstale":"2024-04-26T21:16:56.361784+0000","last_undegraded":"2024-04-26T21:16:56.361784+0000","last_fullsized":"2024-04-26T21:16:56.361784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:54.183612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713804+0000","last_change":"2024-04-26T21:16:03.700284+0000","last_active":"2024-04-26T21:16:56.713804+0000","last_peered":"2024-04-26T21:16:56.713804+0000","last_clean":"2024-04-26T21:16:56.713804+0000","last_became_active":"2024-04-26T21:16:03.699965+0000","last_became_peered":"2024-04-26T21:16:03.699965+0000","last_unstale":"2024-04-26T21:16:56.713804+0000","last_undegraded":"2024-04-26T21:16:56.713804+0000","last_fullsized":"2024-04-26T21:16:56.713804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:40.547296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362469+0000","last_change":"2024-04-26T21:16:03.701370+0000","last_active":"2024-04-26T21:16:56.362469+0000","last_peered":"2024-04-26T21:16:56.362469+0000","last_clean":"2024-04-26T21:16:56.362469+0000","last_became_active":"2024-04-26T21:16:03.700585+0000","last_became_peered":"2024-04-26T21:16:03.700585+0000","last_unstale":"2024-04-26T21:16:56.362469+0000","last_undegraded":"2024-04-26T21:16:56.362469+0000","last_fullsized":"2024-04-26T21:16:56.362469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.348009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360613+0000","last_change":"2024-04-26T21:16:03.281141+0000","last_active":"2024-04-26T21:16:56.360613+0000","last_peered":"2024-04-26T21:16:56.360613+0000","last_clean":"2024-04-26T21:16:56.360613+0000","last_became_active":"2024-04-26T21:16:03.280812+0000","last_became_peered":"2024-04-26T21:16:03.280812+0000","last_unstale":"2024-04-26T21:16:56.360613+0000","last_undegraded":"2024-04-26T21:16:56.360613+0000","last_fullsized":"2024-04-26T21:16:56.360613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:11.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362156+0000","last_change":"2024-04-26T21:16:03.711312+0000","last_active":"2024-04-26T21:16:56.362156+0000","last_peered":"2024-04-26T21:16:56.362156+0000","last_clean":"2024-04-26T21:16:56.362156+0000","last_became_active":"2024-04-26T21:16:03.711026+0000","last_became_peered":"2024-04-26T21:16:03.711026+0000","last_unstale":"2024-04-26T21:16:56.362156+0000","last_undegraded":"2024-04-26T21:16:56.362156+0000","last_fullsized":"2024-04-26T21:16:56.362156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:10.185877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359923+0000","last_change":"2024-04-26T21:16:03.273428+0000","last_active":"2024-04-26T21:16:56.359923+0000","last_peered":"2024-04-26T21:16:56.359923+0000","last_clean":"2024-04-26T21:16:56.359923+0000","last_became_active":"2024-04-26T21:16:03.272854+0000","last_became_peered":"2024-04-26T21:16:03.272854+0000","last_unstale":"2024-04-26T21:16:56.359923+0000","last_undegraded":"2024-04-26T21:16:56.359923+0000","last_fullsized":"2024-04-26T21:16:56.359923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:48.735059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361752+0000","last_change":"2024-04-26T21:14:50.729892+0000","last_active":"2024-04-26T21:16:56.361752+0000","last_peered":"2024-04-26T21:16:56.361752+0000","last_clean":"2024-04-26T21:16:56.361752+0000","last_became_active":"2024-04-26T21:14:50.729430+0000","last_became_peered":"2024-04-26T21:14:50.729430+0000","last_unstale":"2024-04-26T21:16:56.361752+0000","last_undegraded":"2024-04-26T21:16:56.361752+0000","last_fullsized":"2024-04-26T21:16:56.361752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:22.039289+0000","last_clean_scrub_stamp":"2024-04-26T21:11:22.039289+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:13.510971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360509+0000","last_change":"2024-04-26T21:16:03.700702+0000","last_active":"2024-04-26T21:16:56.360509+0000","last_peered":"2024-04-26T21:16:56.360509+0000","last_clean":"2024-04-26T21:16:56.360509+0000","last_became_active":"2024-04-26T21:16:03.700412+0000","last_became_peered":"2024-04-26T21:16:03.700412+0000","last_unstale":"2024-04-26T21:16:56.360509+0000","last_undegraded":"2024-04-26T21:16:56.360509+0000","last_fullsized":"2024-04-26T21:16:56.360509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:49.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361896+0000","last_change":"2024-04-26T21:16:03.271730+0000","last_active":"2024-04-26T21:16:56.361896+0000","last_peered":"2024-04-26T21:16:56.361896+0000","last_clean":"2024-04-26T21:16:56.361896+0000","last_became_active":"2024-04-26T21:16:03.271401+0000","last_became_peered":"2024-04-26T21:16:03.271401+0000","last_unstale":"2024-04-26T21:16:56.361896+0000","last_undegraded":"2024-04-26T21:16:56.361896+0000","last_fullsized":"2024-04-26T21:16:56.361896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:16.469022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362223+0000","last_change":"2024-04-26T21:16:03.276229+0000","last_active":"2024-04-26T21:16:56.362223+0000","last_peered":"2024-04-26T21:16:56.362223+0000","last_clean":"2024-04-26T21:16:56.362223+0000","last_became_active":"2024-04-26T21:16:03.275910+0000","last_became_peered":"2024-04-26T21:16:03.275910+0000","last_unstale":"2024-04-26T21:16:56.362223+0000","last_undegraded":"2024-04-26T21:16:56.362223+0000","last_fullsized":"2024-04-26T21:16:56.362223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:03.932254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361647+0000","last_change":"2024-04-26T21:16:03.698293+0000","last_active":"2024-04-26T21:16:56.361647+0000","last_peered":"2024-04-26T21:16:56.361647+0000","last_clean":"2024-04-26T21:16:56.361647+0000","last_became_active":"2024-04-26T21:16:03.697935+0000","last_became_peered":"2024-04-26T21:16:03.697935+0000","last_unstale":"2024-04-26T21:16:56.361647+0000","last_undegraded":"2024-04-26T21:16:56.361647+0000","last_fullsized":"2024-04-26T21:16:56.361647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:54.292921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362638+0000","last_change":"2024-04-26T21:16:03.275162+0000","last_active":"2024-04-26T21:16:56.362638+0000","last_peered":"2024-04-26T21:16:56.362638+0000","last_clean":"2024-04-26T21:16:56.362638+0000","last_became_active":"2024-04-26T21:16:03.274791+0000","last_became_peered":"2024-04-26T21:16:03.274791+0000","last_unstale":"2024-04-26T21:16:56.362638+0000","last_undegraded":"2024-04-26T21:16:56.362638+0000","last_fullsized":"2024-04-26T21:16:56.362638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:15.583747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364592+0000","last_change":"2024-04-26T21:16:03.701473+0000","last_active":"2024-04-26T21:16:56.364592+0000","last_peered":"2024-04-26T21:16:56.364592+0000","last_clean":"2024-04-26T21:16:56.364592+0000","last_became_active":"2024-04-26T21:16:03.701070+0000","last_became_peered":"2024-04-26T21:16:03.701070+0000","last_unstale":"2024-04-26T21:16:56.364592+0000","last_undegraded":"2024-04-26T21:16:56.364592+0000","last_fullsized":"2024-04-26T21:16:56.364592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:25.373301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360117+0000","last_change":"2024-04-26T21:16:03.271036+0000","last_active":"2024-04-26T21:16:56.360117+0000","last_peered":"2024-04-26T21:16:56.360117+0000","last_clean":"2024-04-26T21:16:56.360117+0000","last_became_active":"2024-04-26T21:16:03.270686+0000","last_became_peered":"2024-04-26T21:16:03.270686+0000","last_unstale":"2024-04-26T21:16:56.360117+0000","last_undegraded":"2024-04-26T21:16:56.360117+0000","last_fullsized":"2024-04-26T21:16:56.360117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:21.025757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.368114+0000","last_change":"2024-04-26T21:16:03.705314+0000","last_active":"2024-04-26T21:16:56.368114+0000","last_peered":"2024-04-26T21:16:56.368114+0000","last_clean":"2024-04-26T21:16:56.368114+0000","last_became_active":"2024-04-26T21:16:03.704952+0000","last_became_peered":"2024-04-26T21:16:03.704952+0000","last_unstale":"2024-04-26T21:16:56.368114+0000","last_undegraded":"2024-04-26T21:16:56.368114+0000","last_fullsized":"2024-04-26T21:16:56.368114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:57.363909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713329+0000","last_change":"2024-04-26T21:16:03.280995+0000","last_active":"2024-04-26T21:16:56.713329+0000","last_peered":"2024-04-26T21:16:56.713329+0000","last_clean":"2024-04-26T21:16:56.713329+0000","last_became_active":"2024-04-26T21:16:03.280615+0000","last_became_peered":"2024-04-26T21:16:03.280615+0000","last_unstale":"2024-04-26T21:16:56.713329+0000","last_undegraded":"2024-04-26T21:16:56.713329+0000","last_fullsized":"2024-04-26T21:16:56.713329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:32.373860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362097+0000","last_change":"2024-04-26T21:16:03.701218+0000","last_active":"2024-04-26T21:16:56.362097+0000","last_peered":"2024-04-26T21:16:56.362097+0000","last_clean":"2024-04-26T21:16:56.362097+0000","last_became_active":"2024-04-26T21:16:03.700895+0000","last_became_peered":"2024-04-26T21:16:03.700895+0000","last_unstale":"2024-04-26T21:16:56.362097+0000","last_undegraded":"2024-04-26T21:16:56.362097+0000","last_fullsized":"2024-04-26T21:16:56.362097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:44.104683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362857+0000","last_change":"2024-04-26T21:16:03.277235+0000","last_active":"2024-04-26T21:16:56.362857+0000","last_peered":"2024-04-26T21:16:56.362857+0000","last_clean":"2024-04-26T21:16:56.362857+0000","last_became_active":"2024-04-26T21:16:03.276901+0000","last_became_peered":"2024-04-26T21:16:03.276901+0000","last_unstale":"2024-04-26T21:16:56.362857+0000","last_undegraded":"2024-04-26T21:16:56.362857+0000","last_fullsized":"2024-04-26T21:16:56.362857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:16.297883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359135+0000","last_change":"2024-04-26T21:16:03.704318+0000","last_active":"2024-04-26T21:16:56.359135+0000","last_peered":"2024-04-26T21:16:56.359135+0000","last_clean":"2024-04-26T21:16:56.359135+0000","last_became_active":"2024-04-26T21:16:03.703907+0000","last_became_peered":"2024-04-26T21:16:03.703907+0000","last_unstale":"2024-04-26T21:16:56.359135+0000","last_undegraded":"2024-04-26T21:16:56.359135+0000","last_fullsized":"2024-04-26T21:16:56.359135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:16.431246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372105+0000","last_change":"2024-04-26T21:16:56.371135+0000","last_active":"2024-04-26T21:16:56.372105+0000","last_peered":"2024-04-26T21:16:56.372105+0000","last_clean":"2024-04-26T21:16:56.372105+0000","last_became_active":"2024-04-26T21:16:56.370666+0000","last_became_peered":"2024-04-26T21:16:56.370666+0000","last_unstale":"2024-04-26T21:16:56.372105+0000","last_undegraded":"2024-04-26T21:16:56.372105+0000","last_fullsized":"2024-04-26T21:16:56.372105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:05.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362114+0000","last_change":"2024-04-26T21:16:03.705026+0000","last_active":"2024-04-26T21:16:56.362114+0000","last_peered":"2024-04-26T21:16:56.362114+0000","last_clean":"2024-04-26T21:16:56.362114+0000","last_became_active":"2024-04-26T21:16:03.703336+0000","last_became_peered":"2024-04-26T21:16:03.703336+0000","last_unstale":"2024-04-26T21:16:56.362114+0000","last_undegraded":"2024-04-26T21:16:56.362114+0000","last_fullsized":"2024-04-26T21:16:56.362114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:11.746362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362335+0000","last_change":"2024-04-26T21:16:03.279263+0000","last_active":"2024-04-26T21:16:56.362335+0000","last_peered":"2024-04-26T21:16:56.362335+0000","last_clean":"2024-04-26T21:16:56.362335+0000","last_became_active":"2024-04-26T21:16:03.278948+0000","last_became_peered":"2024-04-26T21:16:03.278948+0000","last_unstale":"2024-04-26T21:16:56.362335+0000","last_undegraded":"2024-04-26T21:16:56.362335+0000","last_fullsized":"2024-04-26T21:16:56.362335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:39.078346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362036+0000","last_change":"2024-04-26T21:16:03.702913+0000","last_active":"2024-04-26T21:16:56.362036+0000","last_peered":"2024-04-26T21:16:56.362036+0000","last_clean":"2024-04-26T21:16:56.362036+0000","last_became_active":"2024-04-26T21:16:03.702507+0000","last_became_peered":"2024-04-26T21:16:03.702507+0000","last_unstale":"2024-04-26T21:16:56.362036+0000","last_undegraded":"2024-04-26T21:16:56.362036+0000","last_fullsized":"2024-04-26T21:16:56.362036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:51.846983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364220+0000","last_change":"2024-04-26T21:16:03.281898+0000","last_active":"2024-04-26T21:16:56.364220+0000","last_peered":"2024-04-26T21:16:56.364220+0000","last_clean":"2024-04-26T21:16:56.364220+0000","last_became_active":"2024-04-26T21:16:03.281544+0000","last_became_peered":"2024-04-26T21:16:03.281544+0000","last_unstale":"2024-04-26T21:16:56.364220+0000","last_undegraded":"2024-04-26T21:16:56.364220+0000","last_fullsized":"2024-04-26T21:16:56.364220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:39.061050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362089+0000","last_change":"2024-04-26T21:16:03.703006+0000","last_active":"2024-04-26T21:16:56.362089+0000","last_peered":"2024-04-26T21:16:56.362089+0000","last_clean":"2024-04-26T21:16:56.362089+0000","last_became_active":"2024-04-26T21:16:03.702705+0000","last_became_peered":"2024-04-26T21:16:03.702705+0000","last_unstale":"2024-04-26T21:16:56.362089+0000","last_undegraded":"2024-04-26T21:16:56.362089+0000","last_fullsized":"2024-04-26T21:16:56.362089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:11.885993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.364037+0000","last_change":"2024-04-26T21:16:03.280891+0000","last_active":"2024-04-26T21:16:56.364037+0000","last_peered":"2024-04-26T21:16:56.364037+0000","last_clean":"2024-04-26T21:16:56.364037+0000","last_became_active":"2024-04-26T21:16:03.280577+0000","last_became_peered":"2024-04-26T21:16:03.280577+0000","last_unstale":"2024-04-26T21:16:56.364037+0000","last_undegraded":"2024-04-26T21:16:56.364037+0000","last_fullsized":"2024-04-26T21:16:56.364037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:28.698997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.378258+0000","last_change":"2024-04-26T21:16:56.378258+0000","last_active":"2024-04-26T21:16:56.378258+0000","last_peered":"2024-04-26T21:16:56.378258+0000","last_clean":"2024-04-26T21:16:56.378258+0000","last_became_active":"2024-04-26T21:16:56.371874+0000","last_became_peered":"2024-04-26T21:16:56.371874+0000","last_unstale":"2024-04-26T21:16:56.378258+0000","last_undegraded":"2024-04-26T21:16:56.378258+0000","last_fullsized":"2024-04-26T21:16:56.378258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:56.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.360941+0000","last_change":"2024-04-26T21:16:03.276651+0000","last_active":"2024-04-26T21:16:56.360941+0000","last_peered":"2024-04-26T21:16:56.360941+0000","last_clean":"2024-04-26T21:16:56.360941+0000","last_became_active":"2024-04-26T21:16:03.276347+0000","last_became_peered":"2024-04-26T21:16:03.276347+0000","last_unstale":"2024-04-26T21:16:56.360941+0000","last_undegraded":"2024-04-26T21:16:56.360941+0000","last_fullsized":"2024-04-26T21:16:56.360941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:02.246053+0000","last_clean_scrub_stamp":"2024-04-26T21:16:02.246053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:48.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362100+0000","last_change":"2024-04-26T21:16:03.705647+0000","last_active":"2024-04-26T21:16:56.362100+0000","last_peered":"2024-04-26T21:16:56.362100+0000","last_clean":"2024-04-26T21:16:56.362100+0000","last_became_active":"2024-04-26T21:16:03.705251+0000","last_became_peered":"2024-04-26T21:16:03.705251+0000","last_unstale":"2024-04-26T21:16:56.362100+0000","last_undegraded":"2024-04-26T21:16:56.362100+0000","last_fullsized":"2024-04-26T21:16:56.362100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:17.427221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359241+0000","last_change":"2024-04-26T21:16:03.701759+0000","last_active":"2024-04-26T21:16:56.359241+0000","last_peered":"2024-04-26T21:16:56.359241+0000","last_clean":"2024-04-26T21:16:56.359241+0000","last_became_active":"2024-04-26T21:16:03.701382+0000","last_became_peered":"2024-04-26T21:16:03.701382+0000","last_unstale":"2024-04-26T21:16:56.359241+0000","last_undegraded":"2024-04-26T21:16:56.359241+0000","last_fullsized":"2024-04-26T21:16:56.359241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:27.789455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.372345+0000","last_change":"2024-04-26T21:16:56.372345+0000","last_active":"2024-04-26T21:16:56.372345+0000","last_peered":"2024-04-26T21:16:56.372345+0000","last_clean":"2024-04-26T21:16:56.372345+0000","last_became_active":"2024-04-26T21:16:56.371900+0000","last_became_peered":"2024-04-26T21:16:56.371900+0000","last_unstale":"2024-04-26T21:16:56.372345+0000","last_undegraded":"2024-04-26T21:16:56.372345+0000","last_fullsized":"2024-04-26T21:16:56.372345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:42.192142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361562+0000","last_change":"2024-04-26T21:16:03.702012+0000","last_active":"2024-04-26T21:16:56.361562+0000","last_peered":"2024-04-26T21:16:56.361562+0000","last_clean":"2024-04-26T21:16:56.361562+0000","last_became_active":"2024-04-26T21:16:03.701066+0000","last_became_peered":"2024-04-26T21:16:03.701066+0000","last_unstale":"2024-04-26T21:16:56.361562+0000","last_undegraded":"2024-04-26T21:16:56.361562+0000","last_fullsized":"2024-04-26T21:16:56.361562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:45.337320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.367378+0000","last_change":"2024-04-26T21:16:03.702640+0000","last_active":"2024-04-26T21:16:56.367378+0000","last_peered":"2024-04-26T21:16:56.367378+0000","last_clean":"2024-04-26T21:16:56.367378+0000","last_became_active":"2024-04-26T21:16:03.702339+0000","last_became_peered":"2024-04-26T21:16:03.702339+0000","last_unstale":"2024-04-26T21:16:56.367378+0000","last_undegraded":"2024-04-26T21:16:56.367378+0000","last_fullsized":"2024-04-26T21:16:56.367378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:24.083734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361731+0000","last_change":"2024-04-26T21:16:03.708369+0000","last_active":"2024-04-26T21:16:56.361731+0000","last_peered":"2024-04-26T21:16:56.361731+0000","last_clean":"2024-04-26T21:16:56.361731+0000","last_became_active":"2024-04-26T21:16:03.708050+0000","last_became_peered":"2024-04-26T21:16:03.708050+0000","last_unstale":"2024-04-26T21:16:56.361731+0000","last_undegraded":"2024-04-26T21:16:56.361731+0000","last_fullsized":"2024-04-26T21:16:56.361731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:58.200043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359681+0000","last_change":"2024-04-26T21:16:03.700362+0000","last_active":"2024-04-26T21:16:56.359681+0000","last_peered":"2024-04-26T21:16:56.359681+0000","last_clean":"2024-04-26T21:16:56.359681+0000","last_became_active":"2024-04-26T21:16:03.700053+0000","last_became_peered":"2024-04-26T21:16:03.700053+0000","last_unstale":"2024-04-26T21:16:56.359681+0000","last_undegraded":"2024-04-26T21:16:56.359681+0000","last_fullsized":"2024-04-26T21:16:56.359681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:23.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.363986+0000","last_change":"2024-04-26T21:16:03.707717+0000","last_active":"2024-04-26T21:16:56.363986+0000","last_peered":"2024-04-26T21:16:56.363986+0000","last_clean":"2024-04-26T21:16:56.363986+0000","last_became_active":"2024-04-26T21:16:03.707343+0000","last_became_peered":"2024-04-26T21:16:03.707343+0000","last_unstale":"2024-04-26T21:16:56.363986+0000","last_undegraded":"2024-04-26T21:16:56.363986+0000","last_fullsized":"2024-04-26T21:16:56.363986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:46.261231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.369892+0000","last_change":"2024-04-26T21:16:56.369892+0000","last_active":"2024-04-26T21:16:56.369892+0000","last_peered":"2024-04-26T21:16:56.369892+0000","last_clean":"2024-04-26T21:16:56.369892+0000","last_became_active":"2024-04-26T21:16:56.369416+0000","last_became_peered":"2024-04-26T21:16:56.369416+0000","last_unstale":"2024-04-26T21:16:56.369892+0000","last_undegraded":"2024-04-26T21:16:56.369892+0000","last_fullsized":"2024-04-26T21:16:56.369892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:45.708969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362186+0000","last_change":"2024-04-26T21:16:03.699537+0000","last_active":"2024-04-26T21:16:56.362186+0000","last_peered":"2024-04-26T21:16:56.362186+0000","last_clean":"2024-04-26T21:16:56.362186+0000","last_became_active":"2024-04-26T21:16:03.699186+0000","last_became_peered":"2024-04-26T21:16:03.699186+0000","last_unstale":"2024-04-26T21:16:56.362186+0000","last_undegraded":"2024-04-26T21:16:56.362186+0000","last_fullsized":"2024-04-26T21:16:56.362186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:34.784929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.362274+0000","last_change":"2024-04-26T21:16:03.709940+0000","last_active":"2024-04-26T21:16:56.362274+0000","last_peered":"2024-04-26T21:16:56.362274+0000","last_clean":"2024-04-26T21:16:56.362274+0000","last_became_active":"2024-04-26T21:16:03.707255+0000","last_became_peered":"2024-04-26T21:16:03.707255+0000","last_unstale":"2024-04-26T21:16:56.362274+0000","last_undegraded":"2024-04-26T21:16:56.362274+0000","last_fullsized":"2024-04-26T21:16:56.362274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:57.652194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.713080+0000","last_change":"2024-04-26T21:16:03.711139+0000","last_active":"2024-04-26T21:16:56.713080+0000","last_peered":"2024-04-26T21:16:56.713080+0000","last_clean":"2024-04-26T21:16:56.713080+0000","last_became_active":"2024-04-26T21:16:03.709720+0000","last_became_peered":"2024-04-26T21:16:03.709720+0000","last_unstale":"2024-04-26T21:16:56.713080+0000","last_undegraded":"2024-04-26T21:16:56.713080+0000","last_fullsized":"2024-04-26T21:16:56.713080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:10.021914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.361384+0000","last_change":"2024-04-26T21:16:03.699140+0000","last_active":"2024-04-26T21:16:56.361384+0000","last_peered":"2024-04-26T21:16:56.361384+0000","last_clean":"2024-04-26T21:16:56.361384+0000","last_became_active":"2024-04-26T21:16:03.698824+0000","last_became_peered":"2024-04-26T21:16:03.698824+0000","last_unstale":"2024-04-26T21:16:56.361384+0000","last_undegraded":"2024-04-26T21:16:56.361384+0000","last_fullsized":"2024-04-26T21:16:56.361384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:55.896278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:16:56.359230+0000","last_change":"2024-04-26T21:16:03.708258+0000","last_active":"2024-04-26T21:16:56.359230+0000","last_peered":"2024-04-26T21:16:56.359230+0000","last_clean":"2024-04-26T21:16:56.359230+0000","last_became_active":"2024-04-26T21:16:03.707932+0000","last_became_peered":"2024-04-26T21:16:03.707932+0000","last_unstale":"2024-04-26T21:16:56.359230+0000","last_undegraded":"2024-04-26T21:16:56.359230+0000","last_fullsized":"2024-04-26T21:16:56.359230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:16:03.250894+0000","last_clean_scrub_stamp":"2024-04-26T21:16:03.250894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:56.758307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244196,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53330,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":243,"num_read_kb":175,"num_write":77,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":160298,"data_compressed":5568,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":458793,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":1102004,"data_compressed":43255,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459492,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":459397,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1049860,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":458910,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":511325,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":458883,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":511667,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":459014,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":458815,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1049460,"data_compressed":41399,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33948,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35588,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":52532,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52874,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":52726,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:18:08.339 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:08.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.474+0000 7efd0f46d640 1 -- 172.21.15.129:0/553414071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 msgr2=0x7efd081057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.474+0000 7efd0f46d640 1 --2- 172.21.15.129:0/553414071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 0x7efd081057f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efd04009900 tx=0x7efd0402a410 comp rx=0 tx=0).stop 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.474+0000 7efd0f46d640 1 -- 172.21.15.129:0/553414071 shutdown_connections 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.474+0000 7efd0f46d640 1 --2- 172.21.15.129:0/553414071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd08108660 0x7efd0810aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.474+0000 7efd0f46d640 1 --2- 172.21.15.129:0/553414071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd08105d30 0x7efd08108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.474+0000 7efd0f46d640 1 --2- 172.21.15.129:0/553414071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 0x7efd081057f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.474+0000 7efd0f46d640 1 -- 172.21.15.129:0/553414071 >> 172.21.15.129:0/553414071 conn(0x7efd080fce10 msgr2=0x7efd080ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.474+0000 7efd0f46d640 1 -- 172.21.15.129:0/553414071 shutdown_connections 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.474+0000 7efd0f46d640 1 -- 172.21.15.129:0/553414071 wait complete. 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0f46d640 1 Processor -- start 2024-04-26T21:18:08.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0f46d640 1 -- start start 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0f46d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 0x7efd08198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0f46d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd08105d30 0x7efd08199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0f46d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd08108660 0x7efd081a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0f46d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810e4b0 con 0x7efd08103400 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0f46d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810e330 con 0x7efd08105d30 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0f46d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810e630 con 0x7efd08108660 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0d209640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 0x7efd08198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0d209640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 0x7efd08198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59910/0 (socket says 172.21.15.129:59910) 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0d209640 1 -- 172.21.15.129:0/3489171519 learned_addr learned my addr 172.21.15.129:0/3489171519 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0ca08640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd08105d30 0x7efd08199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0da0a640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd08108660 0x7efd081a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0d209640 1 -- 172.21.15.129:0/3489171519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd08108660 msgr2=0x7efd081a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0d209640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd08108660 0x7efd081a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0d209640 1 -- 172.21.15.129:0/3489171519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd08105d30 msgr2=0x7efd08199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.475+0000 7efd0d209640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd08105d30 0x7efd08199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.476+0000 7efd0d209640 1 -- 172.21.15.129:0/3489171519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd081a0810 con 0x7efd08103400 2024-04-26T21:18:08.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.476+0000 7efd0d209640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 0x7efd08198c40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efd04005940 tx=0x7efd04004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:08.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.476+0000 7efcfe7fc640 1 -- 172.21.15.129:0/3489171519 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0403f030 con 0x7efd08103400 2024-04-26T21:18:08.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.476+0000 7efcfe7fc640 1 -- 172.21.15.129:0/3489171519 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd04003cb0 con 0x7efd08103400 2024-04-26T21:18:08.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.476+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd081a0aa0 con 0x7efd08103400 2024-04-26T21:18:08.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.476+0000 7efcfe7fc640 1 -- 172.21.15.129:0/3489171519 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04035070 con 0x7efd08103400 2024-04-26T21:18:08.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.476+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd081a0f90 con 0x7efd08103400 2024-04-26T21:18:08.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.477+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd080fe900 con 0x7efd08103400 2024-04-26T21:18:08.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.478+0000 7efcfe7fc640 1 -- 172.21.15.129:0/3489171519 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efd04042030 con 0x7efd08103400 2024-04-26T21:18:08.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.479+0000 7efcfe7fc640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.479+0000 7efcfe7fc640 1 -- 172.21.15.129:0/3489171519 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7efd040f0ef0 con 0x7efd08103400 2024-04-26T21:18:08.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.479+0000 7efd0ca08640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.479+0000 7efd0ca08640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efd0819a100 tx=0x7efcf800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:08.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.483+0000 7efcfe7fc640 1 -- 172.21.15.129:0/3489171519 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efd04003e60 con 0x7efd08103400 2024-04-26T21:18:08.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.660+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd080635d0 con 0x7efd08103400 2024-04-26T21:18:08.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.662+0000 7efcfe7fc640 1 -- 172.21.15.129:0/3489171519 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7efd040bd760 con 0x7efd08103400 2024-04-26T21:18:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:08.663 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 msgr2=0x7efcf40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efd0819a100 tx=0x7efcf800b410 comp rx=0 tx=0).stop 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 msgr2=0x7efd08198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 0x7efd08198c40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efd04005940 tx=0x7efd04004430 comp rx=0 tx=0).stop 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 shutdown_connections 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd08105d30 0x7efd08199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd08108660 0x7efd081a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 --2- 172.21.15.129:0/3489171519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd08103400 0x7efd08198c40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 >> 172.21.15.129:0/3489171519 conn(0x7efd080fce10 msgr2=0x7efd08103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:08.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 shutdown_connections 2024-04-26T21:18:08.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.664+0000 7efd0f46d640 1 -- 172.21.15.129:0/3489171519 wait complete. 2024-04-26T21:18:08.675 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:08 smithi129 ceph-mon[29237]: pgmap v354: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:08.675 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:08 smithi129 ceph-mon[29237]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:18:08.675 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:08 smithi129 ceph-mon[29237]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:18:08.675 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:08 smithi129 ceph-mon[29237]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:18:08.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:16:01.621500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:16:02.592849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:08.677 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:08.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:08 smithi171 ceph-mon[33478]: pgmap v354: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:08 smithi171 ceph-mon[33478]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:18:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:08 smithi171 ceph-mon[33478]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:18:08.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:08 smithi171 ceph-mon[33478]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:18:08.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 -- 172.21.15.129:0/1197662237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8104530 msgr2=0x7f9ba810adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 --2- 172.21.15.129:0/1197662237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8104530 0x7f9ba810adc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009ca0 tx=0x7f9ba002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 -- 172.21.15.129:0/1197662237 shutdown_connections 2024-04-26T21:18:08.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 --2- 172.21.15.129:0/1197662237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba8103b70 0x7f9ba8103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 --2- 172.21.15.129:0/1197662237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8104530 0x7f9ba810adc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 --2- 172.21.15.129:0/1197662237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ba8102980 0x7f9ba8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 -- 172.21.15.129:0/1197662237 >> 172.21.15.129:0/1197662237 conn(0x7f9ba80fdf10 msgr2=0x7f9ba8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 -- 172.21.15.129:0/1197662237 shutdown_connections 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 -- 172.21.15.129:0/1197662237 wait complete. 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 Processor -- start 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.777+0000 7f9bafff0640 1 -- start start 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bafff0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8102980 0x7f9ba8198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bafff0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba8103b70 0x7f9ba81990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bafff0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ba8104530 0x7f9ba81a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bafff0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba810e490 con 0x7f9ba8102980 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bafff0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba810e310 con 0x7f9ba8103b70 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bafff0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba810e610 con 0x7f9ba8104530 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9badd8c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8102980 0x7f9ba8198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9badd8c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8102980 0x7f9ba8198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59928/0 (socket says 172.21.15.129:59928) 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9badd8c640 1 -- 172.21.15.129:0/2057286344 learned_addr learned my addr 172.21.15.129:0/2057286344 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bad58b640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba8103b70 0x7f9ba81990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bae58d640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ba8104530 0x7f9ba81a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9badd8c640 1 -- 172.21.15.129:0/2057286344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ba8104530 msgr2=0x7f9ba81a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9badd8c640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ba8104530 0x7f9ba81a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9badd8c640 1 -- 172.21.15.129:0/2057286344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba8103b70 msgr2=0x7f9ba81990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9badd8c640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba8103b70 0x7f9ba81990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9badd8c640 1 -- 172.21.15.129:0/2057286344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba81a0780 con 0x7f9ba8102980 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9badd8c640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8102980 0x7f9ba8198b60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800d770 tx=0x7f9b9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9b9effd640 1 -- 172.21.15.129:0/2057286344 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98017070 con 0x7f9ba8102980 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9b9effd640 1 -- 172.21.15.129:0/2057286344 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b98009c30 con 0x7f9ba8102980 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba81a0a70 con 0x7f9ba8102980 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9b9effd640 1 -- 172.21.15.129:0/2057286344 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98003cc0 con 0x7f9ba8102980 2024-04-26T21:18:08.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.778+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba81a0fc0 con 0x7f9ba8102980 2024-04-26T21:18:08.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.779+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba8069ba0 con 0x7f9ba8102980 2024-04-26T21:18:08.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.780+0000 7f9b9effd640 1 -- 172.21.15.129:0/2057286344 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9b98003e60 con 0x7f9ba8102980 2024-04-26T21:18:08.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.781+0000 7f9b9effd640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b940a3c40 0x7f9b940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:08.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.781+0000 7f9bad58b640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b940a3c40 0x7f9b940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:08.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.781+0000 7f9b9effd640 1 -- 172.21.15.129:0/2057286344 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9b980cfe80 con 0x7f9ba8102980 2024-04-26T21:18:08.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.781+0000 7f9bad58b640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b940a3c40 0x7f9b940a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9ba819a070 tx=0x7f9ba4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:08.784 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:08 smithi150 ceph-mon[32290]: pgmap v354: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:08.784 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:08 smithi150 ceph-mon[32290]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:18:08.784 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:08 smithi150 ceph-mon[32290]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:18:08.784 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:08 smithi150 ceph-mon[32290]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:18:08.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.785+0000 7f9b9effd640 1 -- 172.21.15.129:0/2057286344 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9b98009de0 con 0x7f9ba8102980 2024-04-26T21:18:08.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.964+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ba802dbf0 con 0x7f9ba8102980 2024-04-26T21:18:08.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.964+0000 7f9b9effd640 1 -- 172.21.15.129:0/2057286344 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f9ba802dbf0 con 0x7f9ba8102980 2024-04-26T21:18:08.966 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b940a3c40 msgr2=0x7f9b940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b940a3c40 0x7f9b940a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9ba819a070 tx=0x7f9ba4005f90 comp rx=0 tx=0).stop 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8102980 msgr2=0x7f9ba8198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8102980 0x7f9ba8198b60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800d770 tx=0x7f9b9800dc30 comp rx=0 tx=0).stop 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 shutdown_connections 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b940a3c40 0x7f9b940a6100 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba8103b70 0x7f9ba81990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ba8104530 0x7f9ba81a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 --2- 172.21.15.129:0/2057286344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ba8102980 0x7f9ba8198b60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.966+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 >> 172.21.15.129:0/2057286344 conn(0x7f9ba80fdf10 msgr2=0x7f9ba8106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.967+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 shutdown_connections 2024-04-26T21:18:08.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:08.967+0000 7f9bafff0640 1 -- 172.21.15.129:0/2057286344 wait complete. 2024-04-26T21:18:08.979 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:18:09.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 -- 172.21.15.129:0/2364781913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 msgr2=0x7f59b4105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 --2- 172.21.15.129:0/2364781913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 0x7f59b4105800 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009900 tx=0x7f59b002a410 comp rx=0 tx=0).stop 2024-04-26T21:18:09.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 -- 172.21.15.129:0/2364781913 shutdown_connections 2024-04-26T21:18:09.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 --2- 172.21.15.129:0/2364781913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59b4108670 0x7f59b410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 --2- 172.21.15.129:0/2364781913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4105d40 0x7f59b4108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 --2- 172.21.15.129:0/2364781913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 0x7f59b4105800 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 -- 172.21.15.129:0/2364781913 >> 172.21.15.129:0/2364781913 conn(0x7f59b40fce20 msgr2=0x7f59b40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 -- 172.21.15.129:0/2364781913 shutdown_connections 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 -- 172.21.15.129:0/2364781913 wait complete. 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.079+0000 7f59bbf67640 1 Processor -- start 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59bbf67640 1 -- start start 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59bbf67640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 0x7f59b4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59bbf67640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59b4105d40 0x7f59b41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59bbf67640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4108670 0x7f59b41a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59bbf67640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f59b410e4f0 con 0x7f59b4103410 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59bbf67640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59b410e370 con 0x7f59b4105d40 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59bbf67640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f59b410e670 con 0x7f59b4108670 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9d03640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 0x7f59b4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9d03640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 0x7f59b4198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59940/0 (socket says 172.21.15.129:59940) 2024-04-26T21:18:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9d03640 1 -- 172.21.15.129:0/1535187797 learned_addr learned my addr 172.21.15.129:0/1535187797 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9502640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59b4105d40 0x7f59b41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9d03640 1 -- 172.21.15.129:0/1535187797 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4108670 msgr2=0x7f59b41a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9d03640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4108670 0x7f59b41a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9d03640 1 -- 172.21.15.129:0/1535187797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59b4105d40 msgr2=0x7f59b41991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9d03640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59b4105d40 0x7f59b41991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9d03640 1 -- 172.21.15.129:0/1535187797 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b41a0890 con 0x7f59b4103410 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9502640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59b4105d40 0x7f59b41991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59b9d03640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 0x7f59b4198c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f59b0005940 tx=0x7f59b0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59aaffd640 1 -- 172.21.15.129:0/1535187797 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b003f030 con 0x7f59b4103410 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59aaffd640 1 -- 172.21.15.129:0/1535187797 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b0003aa0 con 0x7f59b4103410 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b41a0b20 con 0x7f59b4103410 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59aaffd640 1 -- 172.21.15.129:0/1535187797 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0035070 con 0x7f59b4103410 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.080+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b41a1060 con 0x7f59b4103410 2024-04-26T21:18:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.081+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b40fe910 con 0x7f59b4103410 2024-04-26T21:18:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.082+0000 7f59aaffd640 1 -- 172.21.15.129:0/1535187797 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f59b0042030 con 0x7f59b4103410 2024-04-26T21:18:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.083+0000 7f59aaffd640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59a00a3cc0 0x7f59a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.083+0000 7f59aaffd640 1 -- 172.21.15.129:0/1535187797 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f59b00f0ca0 con 0x7f59b4103410 2024-04-26T21:18:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.083+0000 7f59b9502640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59a00a3cc0 0x7f59a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.083+0000 7f59b9502640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59a00a3cc0 0x7f59a00a6180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005fd0 tx=0x7f59a4005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:09.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.088+0000 7f59aaffd640 1 -- 172.21.15.129:0/1535187797 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f59b00bd510 con 0x7f59b4103410 2024-04-26T21:18:09.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.264+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f59b4063530 con 0x7f59b4103410 2024-04-26T21:18:09.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.265+0000 7f59aaffd640 1 -- 172.21.15.129:0/1535187797 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f59b00bce10 con 0x7f59b4103410 2024-04-26T21:18:09.267 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-26T21:18:09.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59a00a3cc0 msgr2=0x7f59a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59a00a3cc0 0x7f59a00a6180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005fd0 tx=0x7f59a4005ea0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 msgr2=0x7f59b4198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 0x7f59b4198c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f59b0005940 tx=0x7f59b0031a50 comp rx=0 tx=0).stop 2024-04-26T21:18:09.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 shutdown_connections 2024-04-26T21:18:09.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59a00a3cc0 0x7f59a00a6180 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59b4105d40 0x7f59b41991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4108670 0x7f59b41a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 --2- 172.21.15.129:0/1535187797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59b4103410 0x7f59b4198c80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 >> 172.21.15.129:0/1535187797 conn(0x7f59b40fce20 msgr2=0x7f59b4103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:09.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 shutdown_connections 2024-04-26T21:18:09.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.267+0000 7f59bbf67640 1 -- 172.21.15.129:0/1535187797 wait complete. 2024-04-26T21:18:09.279 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:18:09.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 -- 172.21.15.129:0/2982890070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0102a50 msgr2=0x7f59c0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2982890070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0102a50 0x7f59c0102ed0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009900 tx=0x7f59ac02a410 comp rx=0 tx=0).stop 2024-04-26T21:18:09.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 -- 172.21.15.129:0/2982890070 shutdown_connections 2024-04-26T21:18:09.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2982890070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59c0103410 0x7f59c0109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2982890070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0102a50 0x7f59c0102ed0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2982890070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59c0101860 0x7f59c0101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 -- 172.21.15.129:0/2982890070 >> 172.21.15.129:0/2982890070 conn(0x7f59c00fcdd0 msgr2=0x7f59c00ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 -- 172.21.15.129:0/2982890070 shutdown_connections 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 -- 172.21.15.129:0/2982890070 wait complete. 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 Processor -- start 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.381+0000 7f59c6f24640 1 -- start start 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c6f24640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0101860 0x7f59c0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c6f24640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59c0102a50 0x7f59c01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c6f24640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59c0103410 0x7f59c01a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c6f24640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f59c010d3f0 con 0x7f59c0101860 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c6f24640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59c010d270 con 0x7f59c0103410 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c6f24640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f59c010d570 con 0x7f59c0102a50 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c4cc0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0101860 0x7f59c0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c4cc0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0101860 0x7f59c0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59958/0 (socket says 172.21.15.129:59958) 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c4cc0640 1 -- 172.21.15.129:0/2893629754 learned_addr learned my addr 172.21.15.129:0/2893629754 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c54c1640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59c0103410 0x7f59c01a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59bffff640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59c0102a50 0x7f59c01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c4cc0640 1 -- 172.21.15.129:0/2893629754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59c0102a50 msgr2=0x7f59c01991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c4cc0640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59c0102a50 0x7f59c01991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c4cc0640 1 -- 172.21.15.129:0/2893629754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59c0103410 msgr2=0x7f59c01a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c4cc0640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59c0103410 0x7f59c01a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c4cc0640 1 -- 172.21.15.129:0/2893629754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c01a0860 con 0x7f59c0101860 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c54c1640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59c0103410 0x7f59c01a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c4cc0640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0101860 0x7f59c0198ca0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f59b400d770 tx=0x7f59b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59bdffb640 1 -- 172.21.15.129:0/2893629754 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4017070 con 0x7f59c0101860 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59bdffb640 1 -- 172.21.15.129:0/2893629754 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b4009c30 con 0x7f59c0101860 2024-04-26T21:18:09.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c01a0b50 con 0x7f59c0101860 2024-04-26T21:18:09.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.382+0000 7f59bdffb640 1 -- 172.21.15.129:0/2893629754 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4003cc0 con 0x7f59c0101860 2024-04-26T21:18:09.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.383+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c01a10a0 con 0x7f59c0101860 2024-04-26T21:18:09.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.384+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c01a12d0 con 0x7f59c0101860 2024-04-26T21:18:09.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.385+0000 7f59bdffb640 1 -- 172.21.15.129:0/2893629754 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f59b4003e60 con 0x7f59c0101860 2024-04-26T21:18:09.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.385+0000 7f59bdffb640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59b00a3c40 0x7f59b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.385+0000 7f59bffff640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59b00a3c40 0x7f59b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.385+0000 7f59bdffb640 1 -- 172.21.15.129:0/2893629754 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f59b40cffd0 con 0x7f59c0101860 2024-04-26T21:18:09.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.386+0000 7f59bffff640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59b00a3c40 0x7f59b00a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005c40 tx=0x7f59ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:09.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.390+0000 7f59bdffb640 1 -- 172.21.15.129:0/2893629754 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f59b409c8e0 con 0x7f59c0101860 2024-04-26T21:18:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.567+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f59c01a1540 con 0x7f59c0101860 2024-04-26T21:18:09.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.568+0000 7f59bdffb640 1 -- 172.21.15.129:0/2893629754 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f59b4009de0 con 0x7f59c0101860 2024-04-26T21:18:09.569 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-26T21:18:09.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59b00a3c40 msgr2=0x7f59b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59b00a3c40 0x7f59b00a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005c40 tx=0x7f59ac000bc0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0101860 msgr2=0x7f59c0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0101860 0x7f59c0198ca0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f59b400d770 tx=0x7f59b400dc30 comp rx=0 tx=0).stop 2024-04-26T21:18:09.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 shutdown_connections 2024-04-26T21:18:09.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59b00a3c40 0x7f59b00a6100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59c0103410 0x7f59c01a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59c0102a50 0x7f59c01991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 --2- 172.21.15.129:0/2893629754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59c0101860 0x7f59c0198ca0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 >> 172.21.15.129:0/2893629754 conn(0x7f59c00fcdd0 msgr2=0x7f59c006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:09.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 shutdown_connections 2024-04-26T21:18:09.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.570+0000 7f59c6f24640 1 -- 172.21.15.129:0/2893629754 wait complete. 2024-04-26T21:18:09.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:09 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3489171519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:09.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:09 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2057286344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:18:09.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:09 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1535187797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:18:09.581 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:09 smithi129 ceph-mon[29237]: pgmap v355: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:18:09.582 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.679+0000 7fb6f467e640 1 -- 172.21.15.129:0/1332761218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 msgr2=0x7fb6ec102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.679+0000 7fb6f467e640 1 --2- 172.21.15.129:0/1332761218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 0x7fb6ec102ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009900 tx=0x7fb6dc02a410 comp rx=0 tx=0).stop 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.679+0000 7fb6f467e640 1 -- 172.21.15.129:0/1332761218 shutdown_connections 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.679+0000 7fb6f467e640 1 --2- 172.21.15.129:0/1332761218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6ec103430 0x7fb6ec109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.679+0000 7fb6f467e640 1 --2- 172.21.15.129:0/1332761218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 0x7fb6ec102ef0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.679+0000 7fb6f467e640 1 --2- 172.21.15.129:0/1332761218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec101880 0x7fb6ec101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.679+0000 7fb6f467e640 1 -- 172.21.15.129:0/1332761218 >> 172.21.15.129:0/1332761218 conn(0x7fb6ec0fce30 msgr2=0x7fb6ec0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.679+0000 7fb6f467e640 1 -- 172.21.15.129:0/1332761218 shutdown_connections 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.679+0000 7fb6f467e640 1 -- 172.21.15.129:0/1332761218 wait complete. 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f467e640 1 Processor -- start 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f467e640 1 -- start start 2024-04-26T21:18:09.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f467e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec101880 0x7fb6ec197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f467e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 0x7fb6ec197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f467e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6ec103430 0x7fb6ec19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f467e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10d390 con 0x7fb6ec102a70 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f467e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10d210 con 0x7fb6ec101880 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f467e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10d510 con 0x7fb6ec103430 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f1c19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 0x7fb6ec197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f1c19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 0x7fb6ec197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59974/0 (socket says 172.21.15.129:59974) 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f1c19640 1 -- 172.21.15.129:0/433480503 learned_addr learned my addr 172.21.15.129:0/433480503 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f241a640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec101880 0x7fb6ec197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f2c1b640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6ec103430 0x7fb6ec19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f1c19640 1 -- 172.21.15.129:0/433480503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6ec103430 msgr2=0x7fb6ec19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f1c19640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6ec103430 0x7fb6ec19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f1c19640 1 -- 172.21.15.129:0/433480503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec101880 msgr2=0x7fb6ec197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f1c19640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec101880 0x7fb6ec197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.680+0000 7fb6f1c19640 1 -- 172.21.15.129:0/433480503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec19f690 con 0x7fb6ec102a70 2024-04-26T21:18:09.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.681+0000 7fb6f1c19640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 0x7fb6ec197fb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005940 tx=0x7fb6dc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:09.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.681+0000 7fb6e37fe640 1 -- 172.21.15.129:0/433480503 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc03f030 con 0x7fb6ec102a70 2024-04-26T21:18:09.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.681+0000 7fb6e37fe640 1 -- 172.21.15.129:0/433480503 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6dc003aa0 con 0x7fb6ec102a70 2024-04-26T21:18:09.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.681+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec19f920 con 0x7fb6ec102a70 2024-04-26T21:18:09.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.681+0000 7fb6e37fe640 1 -- 172.21.15.129:0/433480503 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc035070 con 0x7fb6ec102a70 2024-04-26T21:18:09.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.681+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec19fe10 con 0x7fb6ec102a70 2024-04-26T21:18:09.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.682+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec108c20 con 0x7fb6ec102a70 2024-04-26T21:18:09.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.683+0000 7fb6e37fe640 1 -- 172.21.15.129:0/433480503 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb6dc042030 con 0x7fb6ec102a70 2024-04-26T21:18:09.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.683+0000 7fb6e37fe640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb6d80a3c40 0x7fb6d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:09.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.683+0000 7fb6f241a640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb6d80a3c40 0x7fb6d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:09.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.684+0000 7fb6e37fe640 1 -- 172.21.15.129:0/433480503 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb6dc0f0b30 con 0x7fb6ec102a70 2024-04-26T21:18:09.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.684+0000 7fb6f241a640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb6d80a3c40 0x7fb6d80a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec1028d0 tx=0x7fb6e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:09.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.688+0000 7fb6e37fe640 1 -- 172.21.15.129:0/433480503 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb6dc0bcd50 con 0x7fb6ec102a70 2024-04-26T21:18:09.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:09 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3489171519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:09.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:09 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2057286344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:18:09.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:09 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1535187797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:18:09.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:09 smithi171 ceph-mon[33478]: pgmap v355: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:18:09.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:09 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3489171519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:09.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:09 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2057286344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:18:09.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:09 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1535187797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:18:09.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:09 smithi150 ceph-mon[32290]: pgmap v355: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:18:09.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.904+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6ec02dcf0 con 0x7fb6ec102a70 2024-04-26T21:18:09.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.905+0000 7fb6e37fe640 1 -- 172.21.15.129:0/433480503 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7fb6dc0c1050 con 0x7fb6ec102a70 2024-04-26T21:18:09.906 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:09.906 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:18:09.906 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:18:09.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb6d80a3c40 msgr2=0x7fb6d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb6d80a3c40 0x7fb6d80a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec1028d0 tx=0x7fb6e8005f90 comp rx=0 tx=0).stop 2024-04-26T21:18:09.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 msgr2=0x7fb6ec197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:09.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 0x7fb6ec197fb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005940 tx=0x7fb6dc031a50 comp rx=0 tx=0).stop 2024-04-26T21:18:09.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 shutdown_connections 2024-04-26T21:18:09.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fb6d80a3c40 0x7fb6d80a6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6ec103430 0x7fb6ec19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6ec102a70 0x7fb6ec197fb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 --2- 172.21.15.129:0/433480503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec101880 0x7fb6ec197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:09.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 >> 172.21.15.129:0/433480503 conn(0x7fb6ec0fce30 msgr2=0x7fb6ec105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:09.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 shutdown_connections 2024-04-26T21:18:09.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:09.907+0000 7fb6f467e640 1 -- 172.21.15.129:0/433480503 wait complete. 2024-04-26T21:18:09.920 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:10.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.016+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/4139120305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c105d30 msgr2=0x7f7c9c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.016+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/4139120305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c105d30 0x7f7c9c108120 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009900 tx=0x7f7c8c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.016+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/4139120305 shutdown_connections 2024-04-26T21:18:10.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.016+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/4139120305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c9c108660 0x7f7c9c10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.016+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/4139120305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c105d30 0x7f7c9c108120 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/4139120305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c9c103400 0x7f7c9c1057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/4139120305 >> 172.21.15.129:0/4139120305 conn(0x7f7c9c0fce30 msgr2=0x7f7c9c0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/4139120305 shutdown_connections 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/4139120305 wait complete. 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 Processor -- start 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 -- start start 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c103400 0x7f7c9c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c9c105d30 0x7f7c9c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c9c108660 0x7f7c9c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c10e4b0 con 0x7f7c9c103400 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c10e330 con 0x7f7c9c105d30 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca3ea9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c10e630 con 0x7f7c9c108660 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca1c45640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c103400 0x7f7c9c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca1c45640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c103400 0x7f7c9c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59992/0 (socket says 172.21.15.129:59992) 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.017+0000 7f7ca1c45640 1 -- 172.21.15.129:0/1232320719 learned_addr learned my addr 172.21.15.129:0/1232320719 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca1444640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c9c105d30 0x7f7c9c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca1c45640 1 -- 172.21.15.129:0/1232320719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c9c108660 msgr2=0x7f7c9c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca2446640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c9c108660 0x7f7c9c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca1c45640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c9c108660 0x7f7c9c1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca1c45640 1 -- 172.21.15.129:0/1232320719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c9c105d30 msgr2=0x7f7c9c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca1c45640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c9c105d30 0x7f7c9c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca1c45640 1 -- 172.21.15.129:0/1232320719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c1a0830 con 0x7f7c9c103400 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca2446640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c9c108660 0x7f7c9c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca1c45640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c103400 0x7f7c9c198c70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800e9f0 tx=0x7f7c9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7c92ffd640 1 -- 172.21.15.129:0/1232320719 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800cd80 con 0x7f7c9c103400 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7c92ffd640 1 -- 172.21.15.129:0/1232320719 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c980108e0 con 0x7f7c9c103400 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c1a0b20 con 0x7f7c9c103400 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7c92ffd640 1 -- 172.21.15.129:0/1232320719 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98018980 con 0x7f7c9c103400 2024-04-26T21:18:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.018+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c1a10c0 con 0x7f7c9c103400 2024-04-26T21:18:10.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.019+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c0fe920 con 0x7f7c9c103400 2024-04-26T21:18:10.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.020+0000 7f7c92ffd640 1 -- 172.21.15.129:0/1232320719 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7c98010430 con 0x7f7c9c103400 2024-04-26T21:18:10.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.021+0000 7f7c92ffd640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7c880a3cc0 0x7f7c880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.021+0000 7f7ca1444640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7c880a3cc0 0x7f7c880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.021+0000 7f7c92ffd640 1 -- 172.21.15.129:0/1232320719 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7c980d0940 con 0x7f7c9c103400 2024-04-26T21:18:10.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.021+0000 7f7ca1444640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7c880a3cc0 0x7f7c880a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009230 tx=0x7f7c8c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.025+0000 7f7c92ffd640 1 -- 172.21.15.129:0/1232320719 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7c9809d1b0 con 0x7f7c9c103400 2024-04-26T21:18:10.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.202+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c9c02dd60 con 0x7f7c9c103400 2024-04-26T21:18:10.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.204+0000 7f7c92ffd640 1 -- 172.21.15.129:0/1232320719 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7f7c9809cab0 con 0x7f7c9c103400 2024-04-26T21:18:10.205 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:10.205 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:10.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7c880a3cc0 msgr2=0x7f7c880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7c880a3cc0 0x7f7c880a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009230 tx=0x7f7c8c0314a0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c103400 msgr2=0x7f7c9c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c103400 0x7f7c9c198c70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800e9f0 tx=0x7f7c9800eeb0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 shutdown_connections 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7c880a3cc0 0x7f7c880a6180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c9c105d30 0x7f7c9c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c9c108660 0x7f7c9c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 --2- 172.21.15.129:0/1232320719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c9c103400 0x7f7c9c198c70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 >> 172.21.15.129:0/1232320719 conn(0x7f7c9c0fce30 msgr2=0x7f7c9c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 shutdown_connections 2024-04-26T21:18:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.206+0000 7f7ca3ea9640 1 -- 172.21.15.129:0/1232320719 wait complete. 2024-04-26T21:18:10.219 DEBUG:tasks.fwd_scrub:Yielding 2024-04-26T21:18:10.219 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T21:18:10.231 INFO:tasks.workunit:Pulling workunits from ref bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:18:10.231 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T21:18:10.231 INFO:tasks.workunit:timeout=3h 2024-04-26T21:18:10.231 INFO:tasks.workunit:cleanup=True 2024-04-26T21:18:10.231 DEBUG:teuthology.orchestra.run.smithi129:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:10.232 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-26T21:18:10.232 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:18:10.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:10.232 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:18:10.247 INFO:teuthology.orchestra.run.smithi129.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:10.247 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T21:18:10.247 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-26T21:18:10.247 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:18:10.247 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T21:18:10.247 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-26 21:16:10.056819350 +0000 2024-04-26T21:18:10.247 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-26 21:16:10.056819350 +0000 2024-04-26T21:18:10.247 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-26 21:18:04.197359243 +0000 2024-04-26T21:18:10.247 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-26 21:16:10.056819350 +0000 2024-04-26T21:18:10.247 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:10.248 DEBUG:teuthology.orchestra.run.smithi129:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T21:18:10.314 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:18:10.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.335+0000 7f7e74112640 1 -- 172.21.15.129:0/3393054102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c102a60 msgr2=0x7f7e6c102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.335+0000 7f7e74112640 1 --2- 172.21.15.129:0/3393054102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c102a60 0x7f7e6c102ee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009900 tx=0x7f7e5c02a410 comp rx=0 tx=0).stop 2024-04-26T21:18:10.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.335+0000 7f7e74112640 1 -- 172.21.15.129:0/3393054102 shutdown_connections 2024-04-26T21:18:10.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.335+0000 7f7e74112640 1 --2- 172.21.15.129:0/3393054102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e6c103420 0x7f7e6c109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.335+0000 7f7e74112640 1 --2- 172.21.15.129:0/3393054102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c102a60 0x7f7e6c102ee0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.335+0000 7f7e74112640 1 --2- 172.21.15.129:0/3393054102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e6c101870 0x7f7e6c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.335+0000 7f7e74112640 1 -- 172.21.15.129:0/3393054102 >> 172.21.15.129:0/3393054102 conn(0x7f7e6c0fcde0 msgr2=0x7f7e6c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 -- 172.21.15.129:0/3393054102 shutdown_connections 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 -- 172.21.15.129:0/3393054102 wait complete. 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 Processor -- start 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 -- start start 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e6c101870 0x7f7e6c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e6c102a60 0x7f7e6c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c103420 0x7f7e6c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c10d3a0 con 0x7f7e6c103420 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c10d220 con 0x7f7e6c101870 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e74112640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c10d520 con 0x7f7e6c102a60 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e726af640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c103420 0x7f7e6c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e726af640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c103420 0x7f7e6c19f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60012/0 (socket says 172.21.15.129:60012) 2024-04-26T21:18:10.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e726af640 1 -- 172.21.15.129:0/4127563563 learned_addr learned my addr 172.21.15.129:0/4127563563 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e71eae640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e6c101870 0x7f7e6c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.336+0000 7f7e716ad640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e6c102a60 0x7f7e6c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e726af640 1 -- 172.21.15.129:0/4127563563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e6c102a60 msgr2=0x7f7e6c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e726af640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e6c102a60 0x7f7e6c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e726af640 1 -- 172.21.15.129:0/4127563563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e6c101870 msgr2=0x7f7e6c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e726af640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e6c101870 0x7f7e6c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e726af640 1 -- 172.21.15.129:0/4127563563 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c19f750 con 0x7f7e6c103420 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e726af640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c103420 0x7f7e6c19f0e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400cad0 tx=0x7f7e6400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e62ffd640 1 -- 172.21.15.129:0/4127563563 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400ae20 con 0x7f7e6c103420 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e62ffd640 1 -- 172.21.15.129:0/4127563563 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e64005020 con 0x7f7e6c103420 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e74112640 1 -- 172.21.15.129:0/4127563563 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c19fa40 con 0x7f7e6c103420 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e62ffd640 1 -- 172.21.15.129:0/4127563563 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400f6b0 con 0x7f7e6c103420 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.337+0000 7f7e74112640 1 -- 172.21.15.129:0/4127563563 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c19ff60 con 0x7f7e6c103420 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e76e20740 1 Processor -- start 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e62ffd640 1 -- 172.21.15.129:0/4127563563 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7e640051d0 con 0x7f7e6c103420 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e76e20740 1 -- start start 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb05f8b0 0x55b2cb05fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb0602a0 0x55b2cb064da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0678a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e76e20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2cafc4b00 con 0x55b2cb065410 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e76e20740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b2caf60920 con 0x55b2cb0602a0 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e76e20740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b2cafc0ac0 con 0x55b2cb05f8b0 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e716ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e716ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0678a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60014/0 (socket says 172.21.15.129:60014) 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e716ad640 1 -- 172.21.15.129:0/4293251262 learned_addr learned my addr 172.21.15.129:0/4293251262 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.340+0000 7f7e71eae640 1 --2- 172.21.15.129:0/4293251262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb0602a0 0x55b2cb064da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e716ad640 1 --2- 172.21.15.129:0/4293251262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb05f8b0 0x55b2cb05fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e62ffd640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7e580a3c40 0x7f7e580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e716ad640 1 -- 172.21.15.129:0/4293251262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb05f8b0 msgr2=0x55b2cb05fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e716ad640 1 --2- 172.21.15.129:0/4293251262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb05f8b0 0x55b2cb05fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e716ad640 1 -- 172.21.15.129:0/4293251262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb0602a0 msgr2=0x55b2cb064da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e716ad640 1 --2- 172.21.15.129:0/4293251262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb0602a0 0x55b2cb064da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e716ad640 1 -- 172.21.15.129:0/4293251262 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2cad9e7e0 con 0x55b2cb065410 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e726af640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7e580a3c40 0x7f7e580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e62ffd640 1 -- 172.21.15.129:0/4127563563 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7e640d0520 con 0x7f7e6c103420 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e716ad640 1 --2- 172.21.15.129:0/4293251262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0678a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c004750 tx=0x7f7e5c0046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37996c1794d4d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e57fff640 1 -- 172.21.15.129:0/4293251262 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c035040 con 0x55b2cb065410 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e57fff640 1 -- 172.21.15.129:0/4293251262 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e5c0035f0 con 0x55b2cb065410 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.341+0000 7f7e57fff640 1 -- 172.21.15.129:0/4293251262 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0044e0 con 0x55b2cb065410 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 -- 172.21.15.129:0/4293251262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 msgr2=0x55b2cb0678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e726af640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7e580a3c40 0x7f7e580a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e6409c440 tx=0x7f7e6406e1d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 --2- 172.21.15.129:0/4293251262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0678a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c004750 tx=0x7f7e5c0046e0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 -- 172.21.15.129:0/4293251262 shutdown_connections 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 --2- 172.21.15.129:0/4293251262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb0602a0 0x55b2cb064da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 --2- 172.21.15.129:0/4293251262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0678a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 --2- 172.21.15.129:0/4293251262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb05f8b0 0x55b2cb05fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 -- 172.21.15.129:0/4293251262 >> 172.21.15.129:0/4293251262 conn(0x55b2cb05af20 msgr2=0x55b2cb05b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 -- 172.21.15.129:0/4293251262 shutdown_connections 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 -- 172.21.15.129:0/4293251262 wait complete. 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 -- start start 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.342+0000 7f7e76e20740 1 Processor -- start 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb05f8b0 0x55b2cb069d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 0x55b2cb06a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb065410 0x55b2cb071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e76e20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb068360 con 0x55b2cb05f8b0 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e76e20740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b2caf5e5d0 con 0x55b2cb065410 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e76e20740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb0684e0 con 0x55b2cb0602a0 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e76e20740 10 client.0 _ll_register_callbacks cb 0x55b2caf4c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb05f8b0 0x55b2cb069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb05f8b0 0x55b2cb069d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60030/0 (socket says 172.21.15.129:60030) 2024-04-26T21:18:10.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 -- 172.21.15.129:0/1618479360 learned_addr learned my addr 172.21.15.129:0/1618479360 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e71eae640 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 0x55b2cb06a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb065410 0x55b2cb071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 -- 172.21.15.129:0/1618479360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 msgr2=0x55b2cb06a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 0x55b2cb06a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 -- 172.21.15.129:0/1618479360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb065410 msgr2=0x55b2cb071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb065410 0x55b2cb071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 -- 172.21.15.129:0/1618479360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2cadfef70 con 0x55b2cb05f8b0 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.343+0000 7f7e716ad640 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb05f8b0 0x55b2cb069d00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c006010 tx=0x7f7e5c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ce5362d650d239 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e557fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e557fa640 1 -- 172.21.15.129:0/1618479360 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c035040 con 0x55b2cb05f8b0 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e557fa640 1 -- 172.21.15.129:0/1618479360 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7e5c003f70 con 0x55b2cb05f8b0 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e76e20740 1 -- 172.21.15.129:0/1618479360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e5c0035f0 con 0x55b2cb05f8b0 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e557fa640 1 -- 172.21.15.129:0/1618479360 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7e5c004170 con 0x55b2cb05f8b0 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e76e20740 10 client.15354 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:10.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e76e20740 1 -- 172.21.15.129:0/1618479360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b2cac96170 con 0x55b2cb05f8b0 2024-04-26T21:18:10.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e557fa640 1 -- 172.21.15.129:0/1618479360 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e5c0035f0 con 0x55b2cb05f8b0 2024-04-26T21:18:10.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e557fa640 1 -- 172.21.15.129:0/1618479360 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f7e5c059450 con 0x55b2cb05f8b0 2024-04-26T21:18:10.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e76e20740 10 client.15354 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:10.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e76e20740 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:10.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e76e20740 20 client.15354 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:10.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e76e20740 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2cb0753e0 0x55b2cb0777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e76e20740 4 client.15354 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:18:10.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e76e20740 1 -- 172.21.15.129:0/1618479360 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2cafd2710 con 0x55b2cb0753e0 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.344+0000 7f7e71eae640 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2cb0753e0 0x55b2cb0777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.345+0000 7f7e557fa640 10 client.15354 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.345+0000 7f7e71eae640 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2cb0753e0 0x55b2cb0777d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ef7c850ff5d9cc server_cookie=fd4988a258307c6b in_seq=0 out_seq=0 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.345+0000 7f7e557fa640 1 -- 172.21.15.129:0/1618479360 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2cafd2710 con 0x55b2cb0753e0 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.345+0000 7f7e557fa640 10 client.15354 handle_command_reply: tid=0 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.345+0000 7f7e76e20740 1 client.15354 shutdown 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 -- 172.21.15.129:0/1618479360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb05f8b0 msgr2=0x55b2cb069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb05f8b0 0x55b2cb069d00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c006010 tx=0x7f7e5c005e40 comp rx=0 tx=0).stop 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 -- 172.21.15.129:0/1618479360 shutdown_connections 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2cb0753e0 0x55b2cb0777d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb065410 0x55b2cb071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e557fa640 0 client.15354 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:10.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 0x55b2cb06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 --2- 172.21.15.129:0/1618479360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb05f8b0 0x55b2cb069d00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 -- 172.21.15.129:0/1618479360 >> 172.21.15.129:0/1618479360 conn(0x55b2cb05af20 msgr2=0x55b2cb05a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 -- 172.21.15.129:0/1618479360 shutdown_connections 2024-04-26T21:18:10.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 1 -- 172.21.15.129:0/1618479360 wait complete. 2024-04-26T21:18:10.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.346+0000 7f7e76e20740 20 client.15354 trim_cache size 0 max 16384 2024-04-26T21:18:10.355 INFO:tasks.workunit.client.0.smithi129.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T21:18:10.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.363+0000 7f7e76e20740 1 Processor -- start 2024-04-26T21:18:10.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.363+0000 7f7e76e20740 1 -- start start 2024-04-26T21:18:10.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 0x55b2cb06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb0602a0 0x55b2cb059f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb065410 0x55b2cb07bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e76e20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1a6d70 con 0x55b2cb0602a0 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e76e20740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1a95f0 con 0x55b2cb05f8b0 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e76e20740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1a73e0 con 0x55b2cb065410 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e71eae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb0602a0 0x55b2cb059f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e71eae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb0602a0 0x55b2cb059f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60044/0 (socket says 172.21.15.129:60044) 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e71eae640 1 -- 172.21.15.129:0/513704449 learned_addr learned my addr 172.21.15.129:0/513704449 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e716ad640 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 0x55b2cb06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e716ad640 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb065410 0x55b2cb07bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e71eae640 1 -- 172.21.15.129:0/513704449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb065410 msgr2=0x55b2cb07bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e71eae640 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb065410 0x55b2cb07bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e71eae640 1 -- 172.21.15.129:0/513704449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 msgr2=0x55b2cb06a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e71eae640 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 0x55b2cb06a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e71eae640 1 -- 172.21.15.129:0/513704449 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2cb05ee40 con 0x55b2cb0602a0 2024-04-26T21:18:10.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e716ad640 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 0x55b2cb06a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e71eae640 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb0602a0 0x55b2cb059f60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800b740 tx=0x7f7e6800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877c5514c97c6bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.364+0000 7f7e577fe640 1 -- 172.21.15.129:0/513704449 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68003d60 con 0x55b2cb0602a0 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e577fe640 1 -- 172.21.15.129:0/513704449 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e680054d0 con 0x55b2cb0602a0 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e577fe640 1 -- 172.21.15.129:0/513704449 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680201b0 con 0x55b2cb0602a0 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e76e20740 1 -- 172.21.15.129:0/513704449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb0602a0 msgr2=0x55b2cb059f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e76e20740 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb0602a0 0x55b2cb059f60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800b740 tx=0x7f7e6800b390 comp rx=0 tx=0).stop 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e76e20740 1 -- 172.21.15.129:0/513704449 shutdown_connections 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e76e20740 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb065410 0x55b2cb07bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e76e20740 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb0602a0 0x55b2cb059f60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e76e20740 1 --2- 172.21.15.129:0/513704449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 0x55b2cb06a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e76e20740 1 -- 172.21.15.129:0/513704449 >> 172.21.15.129:0/513704449 conn(0x55b2cb05af20 msgr2=0x55b2cb059850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e76e20740 1 -- 172.21.15.129:0/513704449 shutdown_connections 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.365+0000 7f7e76e20740 1 -- 172.21.15.129:0/513704449 wait complete. 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e76e20740 1 -- start start 2024-04-26T21:18:10.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e76e20740 1 Processor -- start 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 0x55b2cb0d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 0x55b2cb0d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e76e20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e76e20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1a7ab0 con 0x55b2cb065410 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e76e20740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1a8f20 con 0x55b2cb05f8b0 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e76e20740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1a8180 con 0x55b2cb0602a0 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e76e20740 10 client.0 _ll_register_callbacks cb 0x7f7e5c032c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e716ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e716ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0dcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60060/0 (socket says 172.21.15.129:60060) 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e716ad640 1 -- 172.21.15.129:0/3029454997 learned_addr learned my addr 172.21.15.129:0/3029454997 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e716ad640 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 0x55b2cb0d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e71eae640 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 0x55b2cb0d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e716ad640 1 -- 172.21.15.129:0/3029454997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 msgr2=0x55b2cb0d5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e716ad640 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 0x55b2cb0d5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e716ad640 1 -- 172.21.15.129:0/3029454997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 msgr2=0x55b2cb0d5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e716ad640 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 0x55b2cb0d5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e716ad640 1 -- 172.21.15.129:0/3029454997 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2cb0dd0e0 con 0x55b2cb065410 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.366+0000 7f7e71eae640 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 0x55b2cb0d5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e716ad640 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0dcad0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c02a930 tx=0x7f7e5c02ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bd30ad0c9aae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e557fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e557fa640 1 -- 172.21.15.129:0/3029454997 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c059ce0 con 0x55b2cb065410 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e557fa640 1 -- 172.21.15.129:0/3029454997 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7e5c059450 con 0x55b2cb065410 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e76e20740 1 -- 172.21.15.129:0/3029454997 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e680054d0 con 0x55b2cb065410 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e557fa640 1 -- 172.21.15.129:0/3029454997 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7e5c059650 con 0x55b2cb065410 2024-04-26T21:18:10.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e76e20740 10 client.15360 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e76e20740 1 -- 172.21.15.129:0/3029454997 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b2cb0de420 con 0x55b2cb065410 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e557fa640 1 -- 172.21.15.129:0/3029454997 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e680054d0 con 0x55b2cb065410 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e557fa640 1 -- 172.21.15.129:0/3029454997 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f7e5c03d920 con 0x55b2cb065410 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e76e20740 10 client.15360 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e76e20740 10 client.15360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e76e20740 20 client.15360 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e76e20740 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2cb0dec60 0x55b2cb0e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e76e20740 4 client.15360 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.367+0000 7f7e76e20740 1 -- 172.21.15.129:0/3029454997 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b2cafcc580 con 0x55b2cb0dec60 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.368+0000 7f7e71eae640 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2cb0dec60 0x55b2cb0e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.368+0000 7f7e557fa640 10 client.15360 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.368+0000 7f7e71eae640 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2cb0dec60 0x55b2cb0e1080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc17ebda5c9e6ee9 server_cookie=bf20ef2b19eb3651 in_seq=0 out_seq=0 2024-04-26T21:18:10.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e557fa640 1 -- 172.21.15.129:0/3029454997 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b2cafcc580 con 0x55b2cb0dec60 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e557fa640 10 client.15360 handle_command_reply: tid=0 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 client.15360 shutdown 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 -- 172.21.15.129:0/3029454997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 msgr2=0x55b2cb0dcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0dcad0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c02a930 tx=0x7f7e5c02ab10 comp rx=0 tx=0).stop 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 -- 172.21.15.129:0/3029454997 shutdown_connections 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2cb0dec60 0x55b2cb0e1080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2cb065410 0x55b2cb0dcad0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2cb0602a0 0x55b2cb0d5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e557fa640 0 client.15360 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 --2- 172.21.15.129:0/3029454997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2cb05f8b0 0x55b2cb0d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 -- 172.21.15.129:0/3029454997 >> 172.21.15.129:0/3029454997 conn(0x55b2cb05af20 msgr2=0x55b2cb05a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 -- 172.21.15.129:0/3029454997 shutdown_connections 2024-04-26T21:18:10.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 1 -- 172.21.15.129:0/3029454997 wait complete. 2024-04-26T21:18:10.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.369+0000 7f7e76e20740 20 client.15360 trim_cache size 0 max 16384 2024-04-26T21:18:10.377 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:18:10.377 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:18:10.377 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "43d80cbb-648f-47e1-aeac-a42c2fcf56b2", 2024-04-26T21:18:10.377 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:18:10.377 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:18:10.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 -- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7e580a3c40 msgr2=0x7f7e580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7e580a3c40 0x7f7e580a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e6409c440 tx=0x7f7e6406e1d0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 -- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c103420 msgr2=0x7f7e6c19f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c103420 0x7f7e6c19f0e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400cad0 tx=0x7f7e6400cf90 comp rx=0 tx=0).stop 2024-04-26T21:18:10.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 -- 172.21.15.129:0/4127563563 shutdown_connections 2024-04-26T21:18:10.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f7e580a3c40 0x7f7e580a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c103420 0x7f7e6c19f0e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e6c102a60 0x7f7e6c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 --2- 172.21.15.129:0/4127563563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e6c101870 0x7f7e6c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 -- 172.21.15.129:0/4127563563 >> 172.21.15.129:0/4127563563 conn(0x7f7e6c0fcde0 msgr2=0x7f7e6c105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 -- 172.21.15.129:0/4127563563 shutdown_connections 2024-04-26T21:18:10.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.370+0000 7f7e74112640 1 -- 172.21.15.129:0/4127563563 wait complete. 2024-04-26T21:18:10.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:18:10.382 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:18:10.469 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:10 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2893629754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:18:10.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:10 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/433480503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:18:10.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:10 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1232320719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.527+0000 7f798a39e640 1 -- 172.21.15.129:0/1406826666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101a70 msgr2=0x7f7984101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.527+0000 7f798a39e640 1 --2- 172.21.15.129:0/1406826666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101a70 0x7f7984101ed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7974009900 tx=0x7f797402a410 comp rx=0 tx=0).stop 2024-04-26T21:18:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.527+0000 7f798a39e640 1 -- 172.21.15.129:0/1406826666 shutdown_connections 2024-04-26T21:18:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.527+0000 7f798a39e640 1 --2- 172.21.15.129:0/1406826666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7984102410 0x7f7984102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.527+0000 7f798a39e640 1 --2- 172.21.15.129:0/1406826666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101a70 0x7f7984101ed0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.527+0000 7f798a39e640 1 --2- 172.21.15.129:0/1406826666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7984101110 0x7f7984101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.527+0000 7f798a39e640 1 -- 172.21.15.129:0/1406826666 >> 172.21.15.129:0/1406826666 conn(0x7f79840fccd0 msgr2=0x7f79840ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 -- 172.21.15.129:0/1406826666 shutdown_connections 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 -- 172.21.15.129:0/1406826666 wait complete. 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 Processor -- start 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 -- start start 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101110 0x7f798419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7984101a70 0x7f798419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7984102410 0x7f79841a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7984110530 con 0x7f7984101110 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79841103b0 con 0x7f7984102410 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.528+0000 7f798a39e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f79841106b0 con 0x7f7984101a70 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798939c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101110 0x7f798419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798939c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101110 0x7f798419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60066/0 (socket says 172.21.15.129:60066) 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798939c640 1 -- 172.21.15.129:0/1935773049 learned_addr learned my addr 172.21.15.129:0/1935773049 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f7988b9b640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7984101a70 0x7f798419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f7989b9d640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7984102410 0x7f79841a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798939c640 1 -- 172.21.15.129:0/1935773049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7984101a70 msgr2=0x7f798419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798939c640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7984101a70 0x7f798419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798939c640 1 -- 172.21.15.129:0/1935773049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7984102410 msgr2=0x7f79841a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798939c640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7984102410 0x7f79841a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798939c640 1 -- 172.21.15.129:0/1935773049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79841a2890 con 0x7f7984101110 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798939c640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101110 0x7f798419ac80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f798000d950 tx=0x7f798000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f797a7fc640 1 -- 172.21.15.129:0/1935773049 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980017070 con 0x7f7984101110 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f797a7fc640 1 -- 172.21.15.129:0/1935773049 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7980009c30 con 0x7f7984101110 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f797a7fc640 1 -- 172.21.15.129:0/1935773049 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798000f620 con 0x7f7984101110 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798a39e640 1 -- 172.21.15.129:0/1935773049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79841a2b80 con 0x7f7984101110 2024-04-26T21:18:10.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.529+0000 7f798a39e640 1 -- 172.21.15.129:0/1935773049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79841a3120 con 0x7f7984101110 2024-04-26T21:18:10.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.532+0000 7f797a7fc640 1 -- 172.21.15.129:0/1935773049 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7980010460 con 0x7f7984101110 2024-04-26T21:18:10.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.532+0000 7f798e3f0740 1 Processor -- start 2024-04-26T21:18:10.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.532+0000 7f798e3f0740 1 -- start start 2024-04-26T21:18:10.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.532+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 0x55e98cf534d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.532+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf54700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.532+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 0x55e98cf5b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.532+0000 7f798e3f0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e98ceb7d80 con 0x55e98cf54280 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.532+0000 7f798e3f0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e98ce718b0 con 0x55e98cf530b0 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.532+0000 7f798e3f0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e98ceb3d40 con 0x55e98cf54c40 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f798939c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf54700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f797a7fc640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79700a3cc0 0x7f79700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f798939c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf54700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60076/0 (socket says 172.21.15.129:60076) 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f798939c640 1 -- 172.21.15.129:0/161936689 learned_addr learned my addr 172.21.15.129:0/161936689 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7988b9b640 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 0x55e98cf5b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7989b9d640 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 0x55e98cf534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7989b9d640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79700a3cc0 0x7f79700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f797a7fc640 1 -- 172.21.15.129:0/1935773049 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f79800d0150 con 0x7f7984101110 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7988b9b640 1 -- 172.21.15.129:0/161936689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 msgr2=0x55e98cf534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7988b9b640 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 0x55e98cf534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7988b9b640 1 -- 172.21.15.129:0/161936689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 msgr2=0x55e98cf54700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7988b9b640 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf54700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7988b9b640 1 -- 172.21.15.129:0/161936689 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98ce42a40 con 0x55e98cf54c40 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f798939c640 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf54700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7989b9d640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79700a3cc0 0x7f79700a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55e98cf54180 tx=0x7f797c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f7988b9b640 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 0x55e98cf5b4d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7974005ab0 tx=0x7f7974031b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47fab82f9abadca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f796f7fe640 1 -- 172.21.15.129:0/161936689 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974035030 con 0x55e98cf54c40 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f796f7fe640 1 -- 172.21.15.129:0/161936689 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7974040070 con 0x55e98cf54c40 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.533+0000 7f796f7fe640 1 -- 172.21.15.129:0/161936689 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974003b20 con 0x55e98cf54c40 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 -- 172.21.15.129:0/161936689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 msgr2=0x55e98cf5b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 0x55e98cf5b4d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7974005ab0 tx=0x7f7974031b90 comp rx=0 tx=0).stop 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 -- 172.21.15.129:0/161936689 shutdown_connections 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 0x55e98cf5b4d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf54700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 --2- 172.21.15.129:0/161936689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 0x55e98cf534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 -- 172.21.15.129:0/161936689 >> 172.21.15.129:0/161936689 conn(0x55e98cf4e5a0 msgr2=0x55e98cf4e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 -- 172.21.15.129:0/161936689 shutdown_connections 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 -- 172.21.15.129:0/161936689 wait complete. 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 -- start start 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.534+0000 7f798e3f0740 1 Processor -- start 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf530b0 0x55e98cf5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54280 0x55e98cf5da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54c40 0x55e98cf64a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f798e3f0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e98cf5bbf0 con 0x55e98cf54c40 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f798e3f0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e98ce3ed10 con 0x55e98cf54280 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f798e3f0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e98cf5bd70 con 0x55e98cf530b0 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f798e3f0740 10 client.0 _ll_register_callbacks cb 0x55e98ce6b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:10.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7989b9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54c40 0x55e98cf64a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7989b9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54c40 0x55e98cf64a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60086/0 (socket says 172.21.15.129:60086) 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7989b9d640 1 -- 172.21.15.129:0/3196674121 learned_addr learned my addr 172.21.15.129:0/3196674121 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f798939c640 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf530b0 0x55e98cf5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7988b9b640 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54280 0x55e98cf5da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7989b9d640 1 -- 172.21.15.129:0/3196674121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf530b0 msgr2=0x55e98cf5d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7989b9d640 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf530b0 0x55e98cf5d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7989b9d640 1 -- 172.21.15.129:0/3196674121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54280 msgr2=0x55e98cf5da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7989b9d640 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54280 0x55e98cf5da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7989b9d640 1 -- 172.21.15.129:0/3196674121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98cc937d0 con 0x55e98cf54c40 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7988b9b640 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54280 0x55e98cf5da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f7989b9d640 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54c40 0x55e98cf64a60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f797c00fb00 tx=0x7f797c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c107162bec68018f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.535+0000 7f796cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f796cff9640 1 -- 172.21.15.129:0/3196674121 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c01fd80 con 0x55e98cf54c40 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f796cff9640 1 -- 172.21.15.129:0/3196674121 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f797c0293f0 con 0x55e98cf54c40 2024-04-26T21:18:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798e3f0740 1 -- 172.21.15.129:0/3196674121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7974040070 con 0x55e98cf54c40 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f796cff9640 1 -- 172.21.15.129:0/3196674121 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f797c0295f0 con 0x55e98cf54c40 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798e3f0740 10 client.15372 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798e3f0740 1 -- 172.21.15.129:0/3196674121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e98cbc93f0 con 0x55e98cf54c40 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f796cff9640 1 -- 172.21.15.129:0/3196674121 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7974040070 con 0x55e98cf54c40 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f796cff9640 1 -- 172.21.15.129:0/3196674121 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f797c01c040 con 0x55e98cf54c40 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798e3f0740 10 client.15372 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798e3f0740 10 client.15372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798e3f0740 20 client.15372 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e98cf690c0 0x55e98cf6b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798e3f0740 4 client.15372 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798e3f0740 1 -- 172.21.15.129:0/3196674121 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e98cec5990 con 0x55e98cf690c0 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.536+0000 7f798939c640 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e98cf690c0 0x55e98cf6b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.537+0000 7f796cff9640 10 client.15372 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.537+0000 7f798939c640 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e98cf690c0 0x55e98cf6b500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a63bfbf4c1acd0 server_cookie=fefe1fcebfbff688 in_seq=0 out_seq=0 2024-04-26T21:18:10.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.537+0000 7f796cff9640 1 -- 172.21.15.129:0/3196674121 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e98cec5990 con 0x55e98cf690c0 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.537+0000 7f796cff9640 10 client.15372 handle_command_reply: tid=0 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.537+0000 7f798e3f0740 1 client.15372 shutdown 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.537+0000 7f798e3f0740 1 -- 172.21.15.129:0/3196674121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54c40 msgr2=0x55e98cf64a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.537+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54c40 0x55e98cf64a60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f797c00fb00 tx=0x7f797c004a40 comp rx=0 tx=0).stop 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f798e3f0740 1 -- 172.21.15.129:0/3196674121 shutdown_connections 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e98cf690c0 0x55e98cf6b500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54280 0x55e98cf5da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f796cff9640 0 client.15372 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54c40 0x55e98cf64a60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3196674121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf530b0 0x55e98cf5d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f798e3f0740 1 -- 172.21.15.129:0/3196674121 >> 172.21.15.129:0/3196674121 conn(0x55e98cf4e5a0 msgr2=0x55e98cf4d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f798e3f0740 1 -- 172.21.15.129:0/3196674121 shutdown_connections 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f798e3f0740 1 -- 172.21.15.129:0/3196674121 wait complete. 2024-04-26T21:18:10.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.538+0000 7f798e3f0740 20 client.15372 trim_cache size 0 max 16384 2024-04-26T21:18:10.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798e3f0740 1 Processor -- start 2024-04-26T21:18:10.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798e3f0740 1 -- start start 2024-04-26T21:18:10.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf530b0 0x55e98cf4e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54280 0x55e98cf64a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54c40 0x55e98cf51bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798e3f0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f79841a9dd0 con 0x55e98cf530b0 2024-04-26T21:18:10.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798e3f0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79841ac650 con 0x55e98cf54c40 2024-04-26T21:18:10.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798e3f0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f79841aa440 con 0x55e98cf54280 2024-04-26T21:18:10.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798939c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf530b0 0x55e98cf4e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798939c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf530b0 0x55e98cf4e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60098/0 (socket says 172.21.15.129:60098) 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798939c640 1 -- 172.21.15.129:0/3424541516 learned_addr learned my addr 172.21.15.129:0/3424541516 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f7989b9d640 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54c40 0x55e98cf51bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798939c640 1 -- 172.21.15.129:0/3424541516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54280 msgr2=0x55e98cf64a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f7988b9b640 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54280 0x55e98cf64a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798939c640 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54280 0x55e98cf64a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798939c640 1 -- 172.21.15.129:0/3424541516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54c40 msgr2=0x55e98cf51bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798939c640 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54c40 0x55e98cf51bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798939c640 1 -- 172.21.15.129:0/3424541516 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98cb890f0 con 0x55e98cf530b0 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f7988b9b640 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54280 0x55e98cf64a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f7989b9d640 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54c40 0x55e98cf51bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.555+0000 7f798939c640 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf530b0 0x55e98cf4e980 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7980098740 tx=0x7f79800d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd271c25b0b86433 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f796effd640 1 -- 172.21.15.129:0/3424541516 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980020040 con 0x55e98cf530b0 2024-04-26T21:18:10.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f796effd640 1 -- 172.21.15.129:0/3424541516 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7980096e50 con 0x55e98cf530b0 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f796effd640 1 -- 172.21.15.129:0/3424541516 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980098950 con 0x55e98cf530b0 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f798e3f0740 1 -- 172.21.15.129:0/3424541516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf530b0 msgr2=0x55e98cf4e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf530b0 0x55e98cf4e980 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7980098740 tx=0x7f79800d2040 comp rx=0 tx=0).stop 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f798e3f0740 1 -- 172.21.15.129:0/3424541516 shutdown_connections 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf54c40 0x55e98cf51bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54280 0x55e98cf64a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f798e3f0740 1 --2- 172.21.15.129:0/3424541516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf530b0 0x55e98cf4e980 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f798e3f0740 1 -- 172.21.15.129:0/3424541516 >> 172.21.15.129:0/3424541516 conn(0x55e98cf4e5a0 msgr2=0x55e98cf514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f798e3f0740 1 -- 172.21.15.129:0/3424541516 shutdown_connections 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.556+0000 7f798e3f0740 1 -- 172.21.15.129:0/3424541516 wait complete. 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798e3f0740 1 -- start start 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798e3f0740 1 Processor -- start 2024-04-26T21:18:10.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 0x55e98cf61fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf62510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798e3f0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 0x55e98cf63520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798e3f0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f79841aab10 con 0x55e98cf54280 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798e3f0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79841abf80 con 0x55e98cf530b0 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798e3f0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f79841ab1e0 con 0x55e98cf54c40 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798e3f0740 10 client.0 _ll_register_callbacks cb 0x7f797c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f7988b9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf62510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f7988b9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf62510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60106/0 (socket says 172.21.15.129:60106) 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f7988b9b640 1 -- 172.21.15.129:0/610530014 learned_addr learned my addr 172.21.15.129:0/610530014 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f7989b9d640 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 0x55e98cf63520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f798939c640 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 0x55e98cf61fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f7988b9b640 1 -- 172.21.15.129:0/610530014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 msgr2=0x55e98cf63520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f7988b9b640 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 0x55e98cf63520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f7988b9b640 1 -- 172.21.15.129:0/610530014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 msgr2=0x55e98cf61fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:10.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f7988b9b640 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 0x55e98cf61fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.557+0000 7f7988b9b640 1 -- 172.21.15.129:0/610530014 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98ce784b0 con 0x55e98cf54280 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f7988b9b640 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf62510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f797402a930 tx=0x7f79740317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a5c0bc878b7c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f796cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f796cff9640 1 -- 172.21.15.129:0/610530014 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974035070 con 0x55e98cf54280 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f796cff9640 1 -- 172.21.15.129:0/610530014 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7974004440 con 0x55e98cf54280 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f798e3f0740 1 -- 172.21.15.129:0/610530014 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7980096e50 con 0x55e98cf54280 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f796cff9640 1 -- 172.21.15.129:0/610530014 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7974009520 con 0x55e98cf54280 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f798e3f0740 10 client.15378 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f798e3f0740 1 -- 172.21.15.129:0/610530014 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e98cf64c60 con 0x55e98cf54280 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f796cff9640 1 -- 172.21.15.129:0/610530014 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7980096e50 con 0x55e98cf54280 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f796cff9640 1 -- 172.21.15.129:0/610530014 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f7974057050 con 0x55e98cf54280 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f798e3f0740 10 client.15378 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:10.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f798e3f0740 10 client.15378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f798e3f0740 20 client.15378 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f798e3f0740 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e98cfcfbf0 0x55e98cfd2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f798e3f0740 4 client.15378 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.558+0000 7f798e3f0740 1 -- 172.21.15.129:0/610530014 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e98cebf800 con 0x55e98cfcfbf0 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798939c640 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e98cfcfbf0 0x55e98cfd2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f796cff9640 10 client.15378 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798939c640 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e98cfcfbf0 0x55e98cfd2010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac62950a46eceb8 server_cookie=945fc0a3e343ff25 in_seq=0 out_seq=0 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f796cff9640 1 -- 172.21.15.129:0/610530014 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e98cebf800 con 0x55e98cfcfbf0 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f796cff9640 10 client.15378 handle_command_reply: tid=0 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 client.15378 shutdown 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 -- 172.21.15.129:0/610530014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 msgr2=0x55e98cf62510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf62510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f797402a930 tx=0x7f79740317d0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 -- 172.21.15.129:0/610530014 shutdown_connections 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e98cfcfbf0 0x55e98cfd2010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e98cf54c40 0x55e98cf63520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e98cf54280 0x55e98cf62510 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 --2- 172.21.15.129:0/610530014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e98cf530b0 0x55e98cf61fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 -- 172.21.15.129:0/610530014 >> 172.21.15.129:0/610530014 conn(0x55e98cf4e5a0 msgr2=0x55e98cf51d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f796cff9640 0 client.15378 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 -- 172.21.15.129:0/610530014 shutdown_connections 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 1 -- 172.21.15.129:0/610530014 wait complete. 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.559+0000 7f798e3f0740 20 client.15378 trim_cache size 0 max 16384 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:18:10.568 INFO:teuthology.orchestra.run.smithi129.stdout: "43d80cbb-648f-47e1-aeac-a42c2fcf56b2": { 2024-04-26T21:18:10.569 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:18:10.569 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "43d80cbb-648f-47e1-aeac-a42c2fcf56b2", 2024-04-26T21:18:10.569 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:18:10.569 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:18:10.569 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:18:10.569 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 -- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79700a3cc0 msgr2=0x7f79700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79700a3cc0 0x7f79700a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55e98cf54180 tx=0x7f797c009600 comp rx=0 tx=0).stop 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 -- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101110 msgr2=0x7f798419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101110 0x7f798419ac80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f798000d950 tx=0x7f798000de10 comp rx=0 tx=0).stop 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 -- 172.21.15.129:0/1935773049 shutdown_connections 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79700a3cc0 0x7f79700a6180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7984102410 0x7f79841a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7984101a70 0x7f798419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 --2- 172.21.15.129:0/1935773049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7984101110 0x7f798419ac80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 -- 172.21.15.129:0/1935773049 >> 172.21.15.129:0/1935773049 conn(0x7f79840fccd0 msgr2=0x7f79840fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 -- 172.21.15.129:0/1935773049 shutdown_connections 2024-04-26T21:18:10.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:10.560+0000 7f798a39e640 1 -- 172.21.15.129:0/1935773049 wait complete. 2024-04-26T21:18:10.572 INFO:tasks.cephfs.filesystem:scrub status for tag:43d80cbb-648f-47e1-aeac-a42c2fcf56b2 - {'path': '/', 'tag': '43d80cbb-648f-47e1-aeac-a42c2fcf56b2', 'options': 'recursive,force'} 2024-04-26T21:18:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:10 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2893629754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:18:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:10 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/433480503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:18:10.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:10 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1232320719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:10.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:10 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2893629754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:18:10.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:10 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/433480503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:18:10.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:10 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1232320719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:18:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:11 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:18:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:11 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:11 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:18:11.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:11 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:11.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:11 smithi129 ceph-mon[29237]: pgmap v356: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:18:11.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:11 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:18:11.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:11 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:11.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:11 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:18:11.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:11 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:11.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:11 smithi171 ceph-mon[33478]: pgmap v356: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:18:11.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:11 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:18:11.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:11 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:11.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:11 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:18:11.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:11 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:11.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:11 smithi150 ceph-mon[32290]: pgmap v356: 97 pgs: 97 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:18:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:14 smithi150 ceph-mon[32290]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T21:18:14.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:14 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:18:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:14 smithi129 ceph-mon[29237]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T21:18:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:14 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:18:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:14 smithi171 ceph-mon[33478]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T21:18:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:14 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:18:16.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:16 smithi150 ceph-mon[32290]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T21:18:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:16 smithi129 ceph-mon[29237]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T21:18:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:16 smithi171 ceph-mon[33478]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T21:18:18.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:18 smithi150 ceph-mon[32290]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T21:18:18.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:18 smithi129 ceph-mon[29237]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T21:18:18.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:18 smithi171 ceph-mon[33478]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T21:18:20.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:20 smithi150 ceph-mon[32290]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T21:18:20.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:20 smithi129 ceph-mon[29237]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T21:18:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:20 smithi171 ceph-mon[33478]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T21:18:21.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:21 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:18:21.711 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:21 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:18:21.719 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:21 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:18:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:22 smithi150 ceph-mon[32290]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-26T21:18:22.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:22 smithi129 ceph-mon[29237]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-26T21:18:22.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:22 smithi171 ceph-mon[33478]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:23 smithi129 ceph-mon[29237]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T21:18:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:18:23.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:18:23.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:23 smithi171 ceph-mon[33478]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:18:23.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:23 smithi150 ceph-mon[32290]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T21:18:26.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:26 smithi150 ceph-mon[32290]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:26 smithi129 ceph-mon[29237]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:26 smithi171 ceph-mon[33478]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:28.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:28 smithi150 ceph-mon[32290]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:28.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:28 smithi129 ceph-mon[29237]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:28 smithi171 ceph-mon[33478]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:30.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:30 smithi150 ceph-mon[32290]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:30.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:30 smithi129 ceph-mon[29237]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:30 smithi171 ceph-mon[33478]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:32.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:32 smithi150 ceph-mon[32290]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:32 smithi129 ceph-mon[29237]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:32.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:32 smithi171 ceph-mon[33478]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:34 smithi150 ceph-mon[32290]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:34 smithi129 ceph-mon[29237]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:34.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:34 smithi171 ceph-mon[33478]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:36 smithi150 ceph-mon[32290]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:36.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:36 smithi129 ceph-mon[29237]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:36 smithi171 ceph-mon[33478]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:38 smithi129 ceph-mon[29237]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:38.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:38 smithi171 ceph-mon[33478]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:38.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:38 smithi150 ceph-mon[32290]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:40.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:18:40.574 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:18:40.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.689+0000 7f28309f0640 1 -- 172.21.15.129:0/830939899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c103420 msgr2=0x7f282c109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.689+0000 7f28309f0640 1 --2- 172.21.15.129:0/830939899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c103420 0x7f282c109cb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2824009900 tx=0x7f282402a410 comp rx=0 tx=0).stop 2024-04-26T21:18:40.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.689+0000 7f28309f0640 1 -- 172.21.15.129:0/830939899 shutdown_connections 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.689+0000 7f28309f0640 1 --2- 172.21.15.129:0/830939899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c103420 0x7f282c109cb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.689+0000 7f28309f0640 1 --2- 172.21.15.129:0/830939899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f282c102a60 0x7f282c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.689+0000 7f28309f0640 1 --2- 172.21.15.129:0/830939899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f282c101870 0x7f282c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.689+0000 7f28309f0640 1 -- 172.21.15.129:0/830939899 >> 172.21.15.129:0/830939899 conn(0x7f282c0fcde0 msgr2=0x7f282c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.689+0000 7f28309f0640 1 -- 172.21.15.129:0/830939899 shutdown_connections 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.689+0000 7f28309f0640 1 -- 172.21.15.129:0/830939899 wait complete. 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f28309f0640 1 Processor -- start 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f28309f0640 1 -- start start 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f28309f0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f282c101870 0x7f282c105790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f28309f0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c102a60 0x7f282c103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f28309f0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f282c103420 0x7f282c104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f28309f0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f282c10d3a0 con 0x7f282c102a60 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f28309f0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f282c10d220 con 0x7f282c101870 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f28309f0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f282c10d520 con 0x7f282c103420 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f2829d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c102a60 0x7f282c103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f2829d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c102a60 0x7f282c103e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50936/0 (socket says 172.21.15.129:50936) 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f2829d9b640 1 -- 172.21.15.129:0/3263751726 learned_addr learned my addr 172.21.15.129:0/3263751726 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f282a59c640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f282c101870 0x7f282c105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f282ad9d640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f282c103420 0x7f282c104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f2829d9b640 1 -- 172.21.15.129:0/3263751726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f282c103420 msgr2=0x7f282c104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f2829d9b640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f282c103420 0x7f282c104380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f2829d9b640 1 -- 172.21.15.129:0/3263751726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f282c101870 msgr2=0x7f282c105790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f2829d9b640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f282c101870 0x7f282c105790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f2829d9b640 1 -- 172.21.15.129:0/3263751726 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c104ba0 con 0x7f282c102a60 2024-04-26T21:18:40.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.690+0000 7f282a59c640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f282c101870 0x7f282c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:40.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.691+0000 7f2829d9b640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c102a60 0x7f282c103e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f282000c970 tx=0x7f282000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:40.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.691+0000 7f281b7fe640 1 -- 172.21.15.129:0/3263751726 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2820007bb0 con 0x7f282c102a60 2024-04-26T21:18:40.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.691+0000 7f281b7fe640 1 -- 172.21.15.129:0/3263751726 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2820007d50 con 0x7f282c102a60 2024-04-26T21:18:40.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.691+0000 7f28309f0640 1 -- 172.21.15.129:0/3263751726 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c1a2c60 con 0x7f282c102a60 2024-04-26T21:18:40.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.691+0000 7f281b7fe640 1 -- 172.21.15.129:0/3263751726 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282000f450 con 0x7f282c102a60 2024-04-26T21:18:40.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.691+0000 7f28309f0640 1 -- 172.21.15.129:0/3263751726 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1a3160 con 0x7f282c102a60 2024-04-26T21:18:40.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.693+0000 7f281b7fe640 1 -- 172.21.15.129:0/3263751726 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2820004f20 con 0x7f282c102a60 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.693+0000 7f28337e6740 1 Processor -- start 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.693+0000 7f28337e6740 1 -- start start 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f28337e6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 0x55de17bcf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f28337e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd00b0 0x55de17bd0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f28337e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd0a70 0x55de17bd7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f28337e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55de17b33d80 con 0x55de17bd00b0 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f28337e6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de17aed8b0 con 0x55de17bceee0 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f28337e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55de17b2fd40 con 0x55de17bd0a70 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f281b7fe640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28140a3c40 0x7f28140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd00b0 0x55de17bd0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd00b0 0x55de17bd0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50942/0 (socket says 172.21.15.129:50942) 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282a59c640 1 -- 172.21.15.129:0/202051508 learned_addr learned my addr 172.21.15.129:0/202051508 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282ad9d640 1 --2- 172.21.15.129:0/202051508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 0x55de17bcf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282ad9d640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28140a3c40 0x7f28140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f281b7fe640 1 -- 172.21.15.129:0/3263751726 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f28200d03a0 con 0x7f282c102a60 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f2829d9b640 1 --2- 172.21.15.129:0/202051508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd0a70 0x55de17bd7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282a59c640 1 -- 172.21.15.129:0/202051508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd0a70 msgr2=0x55de17bd7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282a59c640 1 --2- 172.21.15.129:0/202051508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd0a70 0x55de17bd7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282a59c640 1 -- 172.21.15.129:0/202051508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 msgr2=0x55de17bcf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282a59c640 1 --2- 172.21.15.129:0/202051508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 0x55de17bcf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282a59c640 1 -- 172.21.15.129:0/202051508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de17abea40 con 0x55de17bd00b0 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282a59c640 1 --2- 172.21.15.129:0/202051508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd00b0 0x55de17bd0530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f281c008ef0 tx=0x7f281c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae2f72daa1aca30 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.694+0000 7f282ad9d640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28140a3c40 0x7f28140a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2824009230 tx=0x7f2824005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f2818ff9640 1 -- 172.21.15.129:0/202051508 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c003b70 con 0x55de17bd00b0 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f2818ff9640 1 -- 172.21.15.129:0/202051508 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f281c003d10 con 0x55de17bd00b0 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f2818ff9640 1 -- 172.21.15.129:0/202051508 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c00fb70 con 0x55de17bd00b0 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f28337e6740 1 -- 172.21.15.129:0/202051508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd00b0 msgr2=0x55de17bd0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f28337e6740 1 --2- 172.21.15.129:0/202051508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd00b0 0x55de17bd0530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f281c008ef0 tx=0x7f281c00d710 comp rx=0 tx=0).stop 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f28337e6740 1 -- 172.21.15.129:0/202051508 shutdown_connections 2024-04-26T21:18:40.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f28337e6740 1 --2- 172.21.15.129:0/202051508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd0a70 0x55de17bd7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f28337e6740 1 --2- 172.21.15.129:0/202051508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd00b0 0x55de17bd0530 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f28337e6740 1 --2- 172.21.15.129:0/202051508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 0x55de17bcf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f28337e6740 1 -- 172.21.15.129:0/202051508 >> 172.21.15.129:0/202051508 conn(0x55de17bca3d0 msgr2=0x55de17bca7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f28337e6740 1 -- 172.21.15.129:0/202051508 shutdown_connections 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.695+0000 7f28337e6740 1 -- 172.21.15.129:0/202051508 wait complete. 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f28337e6740 1 -- start start 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f28337e6740 1 Processor -- start 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f28337e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bceee0 0x55de17bd9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f28337e6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd00b0 0x55de17bd98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f28337e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17be0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f28337e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55de17bd7a80 con 0x55de17bd0a70 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f28337e6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de17abad10 con 0x55de17bd00b0 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f28337e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55de17bd7c00 con 0x55de17bceee0 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f28337e6740 10 client.0 _ll_register_callbacks cb 0x55de17ae7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f282ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17be0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f2829d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd00b0 0x55de17bd98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f282ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17be0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50952/0 (socket says 172.21.15.129:50952) 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f282ad9d640 1 -- 172.21.15.129:0/2772662729 learned_addr learned my addr 172.21.15.129:0/2772662729 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.696+0000 7f282a59c640 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bceee0 0x55de17bd9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f282ad9d640 1 -- 172.21.15.129:0/2772662729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bceee0 msgr2=0x55de17bd9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f282ad9d640 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bceee0 0x55de17bd9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f282ad9d640 1 -- 172.21.15.129:0/2772662729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd00b0 msgr2=0x55de17bd98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f282ad9d640 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd00b0 0x55de17bd98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f282ad9d640 1 -- 172.21.15.129:0/2772662729 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de1790f7d0 con 0x55de17bd0a70 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f2829d9b640 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd00b0 0x55de17bd98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f282ad9d640 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17be0910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2824033000 tx=0x7f2824009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90a833ede35f835 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f2811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f2811ffb640 1 -- 172.21.15.129:0/2772662729 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824035040 con 0x55de17bd0a70 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f2811ffb640 1 -- 172.21.15.129:0/2772662729 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f2824003d70 con 0x55de17bd0a70 2024-04-26T21:18:40.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f28337e6740 1 -- 172.21.15.129:0/2772662729 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281c003d10 con 0x55de17bd0a70 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f2811ffb640 1 -- 172.21.15.129:0/2772662729 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f28240044d0 con 0x55de17bd0a70 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f28337e6740 10 client.15390 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.697+0000 7f28337e6740 1 -- 172.21.15.129:0/2772662729 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55de178453f0 con 0x55de17bd0a70 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f2811ffb640 1 -- 172.21.15.129:0/2772662729 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f281c003d10 con 0x55de17bd0a70 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f2811ffb640 1 -- 172.21.15.129:0/2772662729 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f282403a070 con 0x55de17bd0a70 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f28337e6740 10 client.15390 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f28337e6740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f28337e6740 20 client.15390 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f28337e6740 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17be4de0 0x55de17be7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f28337e6740 4 client.15390 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f28337e6740 1 -- 172.21.15.129:0/2772662729 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de17b41990 con 0x55de17be4de0 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f282a59c640 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17be4de0 0x55de17be7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f282a59c640 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17be4de0 0x55de17be7240 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4096370409943c01 server_cookie=3873914674bfcf15 in_seq=0 out_seq=0 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.698+0000 7f2811ffb640 10 client.15390 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:40.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f2811ffb640 1 -- 172.21.15.129:0/2772662729 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de17b41990 con 0x55de17be4de0 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f2811ffb640 10 client.15390 handle_command_reply: tid=0 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f28337e6740 1 client.15390 shutdown 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f28337e6740 1 -- 172.21.15.129:0/2772662729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 msgr2=0x55de17be0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f28337e6740 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17be0910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2824033000 tx=0x7f2824009260 comp rx=0 tx=0).stop 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f28337e6740 1 -- 172.21.15.129:0/2772662729 shutdown_connections 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f28337e6740 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17be4de0 0x55de17be7240 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f28337e6740 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd00b0 0x55de17bd98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f28337e6740 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17be0910 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f2811ffb640 0 client.15390 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f28337e6740 1 --2- 172.21.15.129:0/2772662729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bceee0 0x55de17bd9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.699+0000 7f28337e6740 1 -- 172.21.15.129:0/2772662729 >> 172.21.15.129:0/2772662729 conn(0x55de17bca3d0 msgr2=0x55de17bc9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.700+0000 7f28337e6740 1 -- 172.21.15.129:0/2772662729 shutdown_connections 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.700+0000 7f28337e6740 1 -- 172.21.15.129:0/2772662729 wait complete. 2024-04-26T21:18:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.700+0000 7f28337e6740 20 client.15390 trim_cache size 0 max 16384 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f28337e6740 1 Processor -- start 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f28337e6740 1 -- start start 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f28337e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bceee0 0x55de17bc93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f28337e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bc9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f28337e6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd0a70 0x55de17be08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f28337e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1a9f80 con 0x55de17bceee0 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f28337e6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1ac7b0 con 0x55de17bd0a70 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f28337e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1aa5a0 con 0x55de17bd00b0 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f282a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bceee0 0x55de17bc93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f282a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bceee0 0x55de17bc93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50958/0 (socket says 172.21.15.129:50958) 2024-04-26T21:18:40.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f282a59c640 1 -- 172.21.15.129:0/751888283 learned_addr learned my addr 172.21.15.129:0/751888283 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f2829d9b640 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bc9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.717+0000 7f282ad9d640 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd0a70 0x55de17be08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f282a59c640 1 -- 172.21.15.129:0/751888283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 msgr2=0x55de17bc9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f282a59c640 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bc9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f282a59c640 1 -- 172.21.15.129:0/751888283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd0a70 msgr2=0x55de17be08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f282a59c640 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd0a70 0x55de17be08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f282a59c640 1 -- 172.21.15.129:0/751888283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de178050f0 con 0x55de17bceee0 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f282ad9d640 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd0a70 0x55de17be08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f2829d9b640 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bc9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f282a59c640 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bceee0 0x55de17bc93d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f281c008ef0 tx=0x7f281c008250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5304cb53737124b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f2813fff640 1 -- 172.21.15.129:0/751888283 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c00f650 con 0x55de17bceee0 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f2813fff640 1 -- 172.21.15.129:0/751888283 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f281c02c070 con 0x55de17bceee0 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f2813fff640 1 -- 172.21.15.129:0/751888283 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c027730 con 0x55de17bceee0 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f28337e6740 1 -- 172.21.15.129:0/751888283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bceee0 msgr2=0x55de17bc93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f28337e6740 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bceee0 0x55de17bc93d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f281c008ef0 tx=0x7f281c008250 comp rx=0 tx=0).stop 2024-04-26T21:18:40.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f28337e6740 1 -- 172.21.15.129:0/751888283 shutdown_connections 2024-04-26T21:18:40.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f28337e6740 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bd0a70 0x55de17be08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f28337e6740 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bc9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f28337e6740 1 --2- 172.21.15.129:0/751888283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bceee0 0x55de17bc93d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.718+0000 7f28337e6740 1 -- 172.21.15.129:0/751888283 >> 172.21.15.129:0/751888283 conn(0x55de17bca3d0 msgr2=0x55de17bc8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:40.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:40 smithi129 ceph-mon[29237]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:40.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 -- 172.21.15.129:0/751888283 shutdown_connections 2024-04-26T21:18:40.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 -- 172.21.15.129:0/751888283 wait complete. 2024-04-26T21:18:40.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 -- start start 2024-04-26T21:18:40.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 Processor -- start 2024-04-26T21:18:40.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 0x55de17bde000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bde540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17bdf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1aac70 con 0x55de17bd0a70 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1ac0e0 con 0x55de17bceee0 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1ab340 con 0x55de17bd00b0 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f28337e6740 10 client.0 _ll_register_callbacks cb 0x7f28240316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f282ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17bdf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f282ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17bdf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50974/0 (socket says 172.21.15.129:50974) 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f282ad9d640 1 -- 172.21.15.129:0/1760237976 learned_addr learned my addr 172.21.15.129:0/1760237976 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f282a59c640 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 0x55de17bde000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f2829d9b640 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bde540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f282ad9d640 1 -- 172.21.15.129:0/1760237976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 msgr2=0x55de17bde540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f282ad9d640 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bde540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f282ad9d640 1 -- 172.21.15.129:0/1760237976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 msgr2=0x55de17bde000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f282ad9d640 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 0x55de17bde000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f282ad9d640 1 -- 172.21.15.129:0/1760237976 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de17af44b0 con 0x55de17bd0a70 2024-04-26T21:18:40.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.719+0000 7f2829d9b640 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bde540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f282ad9d640 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17bdf500 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2824009900 tx=0x7f28240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1413cfd37113a3e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f2811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f2811ffb640 1 -- 172.21.15.129:0/1760237976 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824004e50 con 0x55de17bd0a70 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f2811ffb640 1 -- 172.21.15.129:0/1760237976 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f2824004ff0 con 0x55de17bd0a70 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f28337e6740 1 -- 172.21.15.129:0/1760237976 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281c02c070 con 0x55de17bd0a70 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f2811ffb640 1 -- 172.21.15.129:0/1760237976 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f28240051f0 con 0x55de17bd0a70 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f28337e6740 10 client.15396 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f28337e6740 1 -- 172.21.15.129:0/1760237976 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55de17be0ba0 con 0x55de17bd0a70 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f2811ffb640 1 -- 172.21.15.129:0/1760237976 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f281c02c070 con 0x55de17bd0a70 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f2811ffb640 1 -- 172.21.15.129:0/1760237976 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f2824068050 con 0x55de17bd0a70 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f28337e6740 10 client.15396 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f28337e6740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f28337e6740 20 client.15396 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f28337e6740 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17c4bab0 0x55de17c4ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f28337e6740 4 client.15396 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:18:40.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.720+0000 7f28337e6740 1 -- 172.21.15.129:0/1760237976 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55de17b3b800 con 0x55de17c4bab0 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f282a59c640 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17c4bab0 0x55de17c4ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f2811ffb640 10 client.15396 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f282a59c640 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17c4bab0 0x55de17c4ded0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567254b2fe657403 server_cookie=14565e059314549a in_seq=0 out_seq=0 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f2811ffb640 1 -- 172.21.15.129:0/1760237976 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55de17b3b800 con 0x55de17c4bab0 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f2811ffb640 10 client.15396 handle_command_reply: tid=0 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 client.15396 shutdown 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 -- 172.21.15.129:0/1760237976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 msgr2=0x55de17bdf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17bdf500 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2824009900 tx=0x7f28240048f0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 -- 172.21.15.129:0/1760237976 shutdown_connections 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17c4bab0 0x55de17c4ded0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17bd0a70 0x55de17bdf500 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17bd00b0 0x55de17bde540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 --2- 172.21.15.129:0/1760237976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17bceee0 0x55de17bde000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f2811ffb640 0 client.15396 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:40.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 -- 172.21.15.129:0/1760237976 >> 172.21.15.129:0/1760237976 conn(0x55de17bca3d0 msgr2=0x55de17bcdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:40.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 -- 172.21.15.129:0/1760237976 shutdown_connections 2024-04-26T21:18:40.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 1 -- 172.21.15.129:0/1760237976 wait complete. 2024-04-26T21:18:40.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.721+0000 7f28337e6740 20 client.15396 trim_cache size 0 max 16384 2024-04-26T21:18:40.730 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:18:40.730 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:18:40.730 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:18:40.730 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:18:40.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 -- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28140a3c40 msgr2=0x7f28140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28140a3c40 0x7f28140a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2824009230 tx=0x7f2824005fd0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 -- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c102a60 msgr2=0x7f282c103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c102a60 0x7f282c103e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f282000c970 tx=0x7f282000ce30 comp rx=0 tx=0).stop 2024-04-26T21:18:40.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 -- 172.21.15.129:0/3263751726 shutdown_connections 2024-04-26T21:18:40.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f28140a3c40 0x7f28140a6100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f282c103420 0x7f282c104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f282c102a60 0x7f282c103e40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 --2- 172.21.15.129:0/3263751726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f282c101870 0x7f282c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 -- 172.21.15.129:0/3263751726 >> 172.21.15.129:0/3263751726 conn(0x7f282c0fcde0 msgr2=0x7f282c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:40.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 -- 172.21.15.129:0/3263751726 shutdown_connections 2024-04-26T21:18:40.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.722+0000 7f28309f0640 1 -- 172.21.15.129:0/3263751726 wait complete. 2024-04-26T21:18:40.734 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:18:40.734 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:40 smithi171 ceph-mon[33478]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:40.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3418078775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0104530 msgr2=0x7fe7d010adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3418078775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0104530 0x7fe7d010adc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009ca0 tx=0x7fe7c802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3418078775 shutdown_connections 2024-04-26T21:18:40.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3418078775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0104530 0x7fe7d010adc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3418078775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7d0103b70 0x7fe7d0103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3418078775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d0102980 0x7fe7d0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3418078775 >> 172.21.15.129:0/3418078775 conn(0x7fe7d00fdf30 msgr2=0x7fe7d0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3418078775 shutdown_connections 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3418078775 wait complete. 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.831+0000 7fe7d5eda640 1 Processor -- start 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7d5eda640 1 -- start start 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7d5eda640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0102980 0x7fe7d0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7d5eda640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7d0103b70 0x7fe7d01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7d5eda640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d0104530 0x7fe7d01a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7d5eda640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d010e4b0 con 0x7fe7d0102980 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7d5eda640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d010e330 con 0x7fe7d0104530 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7d5eda640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d010e630 con 0x7fe7d0103b70 2024-04-26T21:18:40.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0102980 0x7fe7d0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0102980 0x7fe7d0198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50986/0 (socket says 172.21.15.129:50986) 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cf7fe640 1 -- 172.21.15.129:0/3207062770 learned_addr learned my addr 172.21.15.129:0/3207062770 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cffff640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d0104530 0x7fe7d01a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7ceffd640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7d0103b70 0x7fe7d01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cf7fe640 1 -- 172.21.15.129:0/3207062770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7d0103b70 msgr2=0x7fe7d01990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cf7fe640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7d0103b70 0x7fe7d01990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cf7fe640 1 -- 172.21.15.129:0/3207062770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d0104530 msgr2=0x7fe7d01a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cf7fe640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d0104530 0x7fe7d01a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cf7fe640 1 -- 172.21.15.129:0/3207062770 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d01a07a0 con 0x7fe7d0102980 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7cf7fe640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0102980 0x7fe7d0198b80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000d770 tx=0x7fe7c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.832+0000 7fe7ccff9640 1 -- 172.21.15.129:0/3207062770 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0017070 con 0x7fe7d0102980 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.833+0000 7fe7ccff9640 1 -- 172.21.15.129:0/3207062770 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0009c30 con 0x7fe7d0102980 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.833+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d01a0a90 con 0x7fe7d0102980 2024-04-26T21:18:40.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.833+0000 7fe7ccff9640 1 -- 172.21.15.129:0/3207062770 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0003cc0 con 0x7fe7d0102980 2024-04-26T21:18:40.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.833+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d01a0fe0 con 0x7fe7d0102980 2024-04-26T21:18:40.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.834+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d0109d40 con 0x7fe7d0102980 2024-04-26T21:18:40.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.835+0000 7fe7ccff9640 1 -- 172.21.15.129:0/3207062770 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe7c0003e60 con 0x7fe7d0102980 2024-04-26T21:18:40.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.835+0000 7fe7ccff9640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7b80a3c40 0x7fe7b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:40.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.835+0000 7fe7ceffd640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7b80a3c40 0x7fe7b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:40.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.835+0000 7fe7ccff9640 1 -- 172.21.15.129:0/3207062770 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe7c00cfe80 con 0x7fe7d0102980 2024-04-26T21:18:40.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.836+0000 7fe7ceffd640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7b80a3c40 0x7fe7b80a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe7d019a090 tx=0x7fe7c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:40.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:40.840+0000 7fe7ccff9640 1 -- 172.21.15.129:0/3207062770 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe7c0009de0 con 0x7fe7d0102980 2024-04-26T21:18:40.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:40 smithi150 ceph-mon[32290]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:41.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.057+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7d002dad0 con 0x7fe7d0102980 2024-04-26T21:18:41.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.058+0000 7fe7ccff9640 1 -- 172.21.15.129:0/3207062770 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7fe7c00bc070 con 0x7fe7d0102980 2024-04-26T21:18:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:18:41.059 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:18:41.060 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:18:41.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7b80a3c40 msgr2=0x7fe7b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7b80a3c40 0x7fe7b80a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe7d019a090 tx=0x7fe7c4005f90 comp rx=0 tx=0).stop 2024-04-26T21:18:41.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0102980 msgr2=0x7fe7d0198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0102980 0x7fe7d0198b80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000d770 tx=0x7fe7c000dc30 comp rx=0 tx=0).stop 2024-04-26T21:18:41.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 shutdown_connections 2024-04-26T21:18:41.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe7b80a3c40 0x7fe7b80a6100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d0104530 0x7fe7d01a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7d0103b70 0x7fe7d01990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 --2- 172.21.15.129:0/3207062770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7d0102980 0x7fe7d0198b80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 >> 172.21.15.129:0/3207062770 conn(0x7fe7d00fdf30 msgr2=0x7fe7d0106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 shutdown_connections 2024-04-26T21:18:41.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.060+0000 7fe7d5eda640 1 -- 172.21.15.129:0/3207062770 wait complete. 2024-04-26T21:18:41.072 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:18:41.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.171+0000 7f180f7d4640 1 -- 172.21.15.129:0/1148773849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 msgr2=0x7f1808101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.171+0000 7f180f7d4640 1 --2- 172.21.15.129:0/1148773849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 0x7f1808101060 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009930 tx=0x7f17fc02a420 comp rx=0 tx=0).stop 2024-04-26T21:18:41.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 -- 172.21.15.129:0/1148773849 shutdown_connections 2024-04-26T21:18:41.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 --2- 172.21.15.129:0/1148773849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18081045d0 0x7f180810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 --2- 172.21.15.129:0/1148773849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1808103c10 0x7f1808104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 --2- 172.21.15.129:0/1148773849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 0x7f1808101060 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 -- 172.21.15.129:0/1148773849 >> 172.21.15.129:0/1148773849 conn(0x7f18080fc570 msgr2=0x7f18080fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 -- 172.21.15.129:0/1148773849 shutdown_connections 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 -- 172.21.15.129:0/1148773849 wait complete. 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 Processor -- start 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 -- start start 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 0x7f1808198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1808103c10 0x7f18081990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18081045d0 0x7f18081a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f180810e4a0 con 0x7f1808100c40 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f180810e320 con 0x7f18081045d0 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180f7d4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f180810e620 con 0x7f1808103c10 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.172+0000 7f180d570640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 0x7f1808198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180d570640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 0x7f1808198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51006/0 (socket says 172.21.15.129:51006) 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180d570640 1 -- 172.21.15.129:0/3751267886 learned_addr learned my addr 172.21.15.129:0/3751267886 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180cd6f640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1808103c10 0x7f18081990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180dd71640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18081045d0 0x7f18081a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180d570640 1 -- 172.21.15.129:0/3751267886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1808103c10 msgr2=0x7f18081990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180d570640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1808103c10 0x7f18081990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180d570640 1 -- 172.21.15.129:0/3751267886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18081045d0 msgr2=0x7f18081a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180d570640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18081045d0 0x7f18081a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180d570640 1 -- 172.21.15.129:0/3751267886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18081a07d0 con 0x7f1808100c40 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180dd71640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18081045d0 0x7f18081a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180d570640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 0x7f1808198bb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009230 tx=0x7f17fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f17fa7fc640 1 -- 172.21.15.129:0/3751267886 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc03f030 con 0x7f1808100c40 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f17fa7fc640 1 -- 172.21.15.129:0/3751267886 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17fc003cb0 con 0x7f1808100c40 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f17fa7fc640 1 -- 172.21.15.129:0/3751267886 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc035070 con 0x7f1808100c40 2024-04-26T21:18:41.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180f7d4640 1 -- 172.21.15.129:0/3751267886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18081a0a60 con 0x7f1808100c40 2024-04-26T21:18:41.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.173+0000 7f180f7d4640 1 -- 172.21.15.129:0/3751267886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18081a0f50 con 0x7f1808100c40 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.175+0000 7f17fa7fc640 1 -- 172.21.15.129:0/3751267886 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f17fc042030 con 0x7f1808100c40 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f18125cd740 1 Processor -- start 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f18125cd740 1 -- start start 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f18125cd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b34820 0x556ab7b34c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f18125cd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b359f0 0x556ab7b35e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f18125cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b363b0 0x556ab7b3cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f18125cd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ab7a98d80 con 0x556ab7b34820 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f18125cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556ab7a52a20 con 0x556ab7b363b0 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f18125cd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556ab7a94d40 con 0x556ab7b359f0 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f17fa7fc640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17f40a3c40 0x7f17f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180dd71640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b34820 0x556ab7b34c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180dd71640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b34820 0x556ab7b34c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51010/0 (socket says 172.21.15.129:51010) 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180dd71640 1 -- 172.21.15.129:0/3694849235 learned_addr learned my addr 172.21.15.129:0/3694849235 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180dd71640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17f40a3c40 0x7f17f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180d570640 1 --2- 172.21.15.129:0/3694849235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b359f0 0x556ab7b35e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180cd6f640 1 --2- 172.21.15.129:0/3694849235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b363b0 0x556ab7b3cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f17fa7fc640 1 -- 172.21.15.129:0/3751267886 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f17fc0f0ba0 con 0x7f1808100c40 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180dd71640 1 -- 172.21.15.129:0/3694849235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b359f0 msgr2=0x556ab7b35e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180dd71640 1 --2- 172.21.15.129:0/3694849235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b359f0 0x556ab7b35e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180dd71640 1 -- 172.21.15.129:0/3694849235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b363b0 msgr2=0x556ab7b3cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180dd71640 1 --2- 172.21.15.129:0/3694849235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b363b0 0x556ab7b3cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.176+0000 7f180dd71640 1 -- 172.21.15.129:0/3694849235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab7a5dc30 con 0x556ab7b34820 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.177+0000 7f180dd71640 1 --2- 172.21.15.129:0/3694849235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b34820 0x556ab7b34c40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1800012e30 tx=0x7f18000107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782a978607db223d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.177+0000 7f180dd71640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17f40a3c40 0x7f17f40a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1800010ca0 tx=0x7f180001b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.177+0000 7f17f37fe640 1 -- 172.21.15.129:0/3694849235 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800004b20 con 0x556ab7b34820 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.177+0000 7f17f37fe640 1 -- 172.21.15.129:0/3694849235 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1800004cc0 con 0x556ab7b34820 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.177+0000 7f17f37fe640 1 -- 172.21.15.129:0/3694849235 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18000059b0 con 0x556ab7b34820 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.177+0000 7f18125cd740 1 -- 172.21.15.129:0/3694849235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b34820 msgr2=0x556ab7b34c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.177+0000 7f18125cd740 1 --2- 172.21.15.129:0/3694849235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b34820 0x556ab7b34c40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1800012e30 tx=0x7f18000107a0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.177+0000 7f18125cd740 1 -- 172.21.15.129:0/3694849235 shutdown_connections 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.177+0000 7f18125cd740 1 --2- 172.21.15.129:0/3694849235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b363b0 0x556ab7b3cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 --2- 172.21.15.129:0/3694849235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b359f0 0x556ab7b35e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 --2- 172.21.15.129:0/3694849235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b34820 0x556ab7b34c40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 -- 172.21.15.129:0/3694849235 >> 172.21.15.129:0/3694849235 conn(0x556ab7a75910 msgr2=0x556ab7b30120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 -- 172.21.15.129:0/3694849235 shutdown_connections 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 -- 172.21.15.129:0/3694849235 wait complete. 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 -- start start 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 Processor -- start 2024-04-26T21:18:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b3eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b3f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 0x556ab7b46060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ab7b3d360 con 0x556ab7b359f0 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556ab7a1fdc0 con 0x556ab7b34820 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556ab7b3d4e0 con 0x556ab7b363b0 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f18125cd740 10 client.0 _ll_register_callbacks cb 0x556ab7a3b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f180cd6f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b3f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f180cd6f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b3f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51014/0 (socket says 172.21.15.129:51014) 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.178+0000 7f180cd6f640 1 -- 172.21.15.129:0/2877987174 learned_addr learned my addr 172.21.15.129:0/2877987174 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f180d570640 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b3eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f180cd6f640 1 -- 172.21.15.129:0/2877987174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 msgr2=0x556ab7b46060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f180dd71640 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 0x556ab7b46060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f180cd6f640 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 0x556ab7b46060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f180cd6f640 1 -- 172.21.15.129:0/2877987174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 msgr2=0x556ab7b3eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f180cd6f640 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b3eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f180cd6f640 1 -- 172.21.15.129:0/2877987174 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab77aa3f0 con 0x556ab7b359f0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f180d570640 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b3eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f180cd6f640 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b3f090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f180400bf80 tx=0x7f18040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1c47cf64ef5049 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f17f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f17f0ff9640 1 -- 172.21.15.129:0/2877987174 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804004020 con 0x556ab7b359f0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f17f0ff9640 1 -- 172.21.15.129:0/2877987174 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f18040041c0 con 0x556ab7b359f0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f18125cd740 1 -- 172.21.15.129:0/2877987174 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1800004cc0 con 0x556ab7b359f0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f17f0ff9640 1 -- 172.21.15.129:0/2877987174 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f18040043c0 con 0x556ab7b359f0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f18125cd740 10 client.15414 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f18125cd740 1 -- 172.21.15.129:0/2877987174 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556ab78747d0 con 0x556ab7b359f0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f17f0ff9640 1 -- 172.21.15.129:0/2877987174 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1800004cc0 con 0x556ab7b359f0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f17f0ff9640 1 -- 172.21.15.129:0/2877987174 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f1804004ca0 con 0x556ab7b359f0 2024-04-26T21:18:41.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f18125cd740 10 client.15414 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.179+0000 7f18125cd740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.180+0000 7f18125cd740 20 client.15414 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.180+0000 7f18125cd740 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556ab7b4a950 0x556ab7b4ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.180+0000 7f18125cd740 4 client.15414 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.180+0000 7f18125cd740 1 -- 172.21.15.129:0/2877987174 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ab7aa6990 con 0x556ab7b4a950 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.180+0000 7f180d570640 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556ab7b4a950 0x556ab7b4ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.180+0000 7f180d570640 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556ab7b4a950 0x556ab7b4ce10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b330f87fdeb0b7 server_cookie=ca09b084dc24752d in_seq=0 out_seq=0 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.180+0000 7f17f0ff9640 10 client.15414 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f17f0ff9640 1 -- 172.21.15.129:0/2877987174 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ab7aa6990 con 0x556ab7b4a950 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f17f0ff9640 10 client.15414 handle_command_reply: tid=0 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 client.15414 shutdown 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 -- 172.21.15.129:0/2877987174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 msgr2=0x556ab7b3f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b3f090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f180400bf80 tx=0x7f18040118d0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 -- 172.21.15.129:0/2877987174 shutdown_connections 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556ab7b4a950 0x556ab7b4ce10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 0x556ab7b46060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b3f090 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f17f0ff9640 0 client.15414 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 --2- 172.21.15.129:0/2877987174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b3eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 -- 172.21.15.129:0/2877987174 >> 172.21.15.129:0/2877987174 conn(0x556ab7a75910 msgr2=0x556ab7b2f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 -- 172.21.15.129:0/2877987174 shutdown_connections 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 1 -- 172.21.15.129:0/2877987174 wait complete. 2024-04-26T21:18:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.181+0000 7f18125cd740 20 client.15414 trim_cache size 0 max 16384 2024-04-26T21:18:41.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f18125cd740 1 Processor -- start 2024-04-26T21:18:41.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f18125cd740 1 -- start start 2024-04-26T21:18:41.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f18125cd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b34820 0x556ab7b337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f18125cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b359f0 0x556ab7b46020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f18125cd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b363b0 0x556ab7b516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f18125cd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18081a7d30 con 0x556ab7b363b0 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f18125cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18081aa5b0 con 0x556ab7b359f0 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f18125cd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f18081a83a0 con 0x556ab7b34820 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f180dd71640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b363b0 0x556ab7b516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.194+0000 7f180dd71640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b363b0 0x556ab7b516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51024/0 (socket says 172.21.15.129:51024) 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180dd71640 1 -- 172.21.15.129:0/3900215333 learned_addr learned my addr 172.21.15.129:0/3900215333 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180d570640 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b34820 0x556ab7b337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180cd6f640 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b359f0 0x556ab7b46020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180dd71640 1 -- 172.21.15.129:0/3900215333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b34820 msgr2=0x556ab7b337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180dd71640 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b34820 0x556ab7b337a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180dd71640 1 -- 172.21.15.129:0/3900215333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b359f0 msgr2=0x556ab7b46020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180dd71640 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b359f0 0x556ab7b46020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180dd71640 1 -- 172.21.15.129:0/3900215333 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab7b3f9f0 con 0x556ab7b363b0 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180cd6f640 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b359f0 0x556ab7b46020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f180dd71640 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b363b0 0x556ab7b516d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f180000aeb0 tx=0x7f180000aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f344494c43432148 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f17f2ffd640 1 -- 172.21.15.129:0/3900215333 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800012240 con 0x556ab7b363b0 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f17f2ffd640 1 -- 172.21.15.129:0/3900215333 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1800005040 con 0x556ab7b363b0 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.195+0000 7f17f2ffd640 1 -- 172.21.15.129:0/3900215333 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180001d6a0 con 0x556ab7b363b0 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 -- 172.21.15.129:0/3900215333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b363b0 msgr2=0x556ab7b516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b363b0 0x556ab7b516d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f180000aeb0 tx=0x7f180000aee0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 -- 172.21.15.129:0/3900215333 shutdown_connections 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b359f0 0x556ab7b46020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b363b0 0x556ab7b516d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 --2- 172.21.15.129:0/3900215333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b34820 0x556ab7b337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 -- 172.21.15.129:0/3900215333 >> 172.21.15.129:0/3900215333 conn(0x556ab7a75910 msgr2=0x556ab7b33090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 -- 172.21.15.129:0/3900215333 shutdown_connections 2024-04-26T21:18:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 -- 172.21.15.129:0/3900215333 wait complete. 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 -- start start 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 Processor -- start 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b43860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b43da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 0x556ab7b44db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18081a8a70 con 0x556ab7b359f0 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18081a9ee0 con 0x556ab7b34820 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f18081a9140 con 0x556ab7b363b0 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f18125cd740 10 client.0 _ll_register_callbacks cb 0x7f180400bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.196+0000 7f180cd6f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b43da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180cd6f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b43da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51034/0 (socket says 172.21.15.129:51034) 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180cd6f640 1 -- 172.21.15.129:0/1893622706 learned_addr learned my addr 172.21.15.129:0/1893622706 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180dd71640 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 0x556ab7b44db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180d570640 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b43860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180cd6f640 1 -- 172.21.15.129:0/1893622706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 msgr2=0x556ab7b44db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180cd6f640 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 0x556ab7b44db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180cd6f640 1 -- 172.21.15.129:0/1893622706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 msgr2=0x556ab7b43860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180cd6f640 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b43860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180cd6f640 1 -- 172.21.15.129:0/1893622706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab7b453a0 con 0x556ab7b359f0 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180dd71640 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 0x556ab7b44db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180d570640 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b43860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f180cd6f640 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b43da0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1804010810 tx=0x7f1804010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e0e3f513ab0985 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f17f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f17f0ff9640 1 -- 172.21.15.129:0/1893622706 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804010e30 con 0x556ab7b359f0 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f17f0ff9640 1 -- 172.21.15.129:0/1893622706 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f1804005250 con 0x556ab7b359f0 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f18125cd740 1 -- 172.21.15.129:0/1893622706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1800005040 con 0x556ab7b359f0 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f17f0ff9640 1 -- 172.21.15.129:0/1893622706 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f18040054b0 con 0x556ab7b359f0 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f18125cd740 10 client.15420 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f18125cd740 1 -- 172.21.15.129:0/1893622706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556ab7bb0fb0 con 0x556ab7b359f0 2024-04-26T21:18:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.197+0000 7f17f0ff9640 1 -- 172.21.15.129:0/1893622706 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1800005040 con 0x556ab7b359f0 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f17f0ff9640 1 -- 172.21.15.129:0/1893622706 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f18040035f0 con 0x556ab7b359f0 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 10 client.15420 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 10 client.15420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 20 client.15420 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556ab7bb1980 0x556ab7bb3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 4 client.15420 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 1 -- 172.21.15.129:0/1893622706 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ab7aa0800 con 0x556ab7bb1980 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f180d570640 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556ab7bb1980 0x556ab7bb3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f180d570640 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556ab7bb1980 0x556ab7bb3d70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b6d2c2b2b2b1ffd server_cookie=d0228c09d7a8fee0 in_seq=0 out_seq=0 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f17f0ff9640 10 client.15420 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f17f0ff9640 1 -- 172.21.15.129:0/1893622706 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ab7aa0800 con 0x556ab7bb1980 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f17f0ff9640 10 client.15420 handle_command_reply: tid=0 2024-04-26T21:18:41.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 1 client.15420 shutdown 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 1 -- 172.21.15.129:0/1893622706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 msgr2=0x556ab7b43da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b43da0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1804010810 tx=0x7f1804010db0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 1 -- 172.21.15.129:0/1893622706 shutdown_connections 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.198+0000 7f18125cd740 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556ab7bb1980 0x556ab7bb3d70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f18125cd740 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab7b363b0 0x556ab7b44db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f17f0ff9640 0 client.15420 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f18125cd740 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ab7b359f0 0x556ab7b43da0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f18125cd740 1 --2- 172.21.15.129:0/1893622706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ab7b34820 0x556ab7b43860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f18125cd740 1 -- 172.21.15.129:0/1893622706 >> 172.21.15.129:0/1893622706 conn(0x556ab7a75910 msgr2=0x556ab7b33920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f18125cd740 1 -- 172.21.15.129:0/1893622706 shutdown_connections 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f18125cd740 1 -- 172.21.15.129:0/1893622706 wait complete. 2024-04-26T21:18:41.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f18125cd740 20 client.15420 trim_cache size 0 max 16384 2024-04-26T21:18:41.208 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:18:41.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 -- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17f40a3c40 msgr2=0x7f17f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17f40a3c40 0x7f17f40a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1800010ca0 tx=0x7f180001b000 comp rx=0 tx=0).stop 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 -- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 msgr2=0x7f1808198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 0x7f1808198bb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009230 tx=0x7f17fc0042f0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 -- 172.21.15.129:0/3751267886 shutdown_connections 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f17f40a3c40 0x7f17f40a6100 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18081045d0 0x7f18081a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1808103c10 0x7f18081990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 --2- 172.21.15.129:0/3751267886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1808100c40 0x7f1808198bb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 -- 172.21.15.129:0/3751267886 >> 172.21.15.129:0/3751267886 conn(0x7f18080fc570 msgr2=0x7f1808107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 -- 172.21.15.129:0/3751267886 shutdown_connections 2024-04-26T21:18:41.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:41.199+0000 7f180f7d4640 1 -- 172.21.15.129:0/3751267886 wait complete. 2024-04-26T21:18:41.392 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:41 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3207062770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:18:41.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:41 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3207062770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:18:41.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:41 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3207062770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:18:42.212 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:18:42.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:42.212 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:18:42.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.326+0000 7f764b351640 1 -- 172.21.15.129:0/3682627420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644103b70 msgr2=0x7f7644103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.326+0000 7f764b351640 1 --2- 172.21.15.129:0/3682627420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644103b70 0x7f7644103ff0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7634009900 tx=0x7f763402a410 comp rx=0 tx=0).stop 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.326+0000 7f764b351640 1 -- 172.21.15.129:0/3682627420 shutdown_connections 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.326+0000 7f764b351640 1 --2- 172.21.15.129:0/3682627420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7644104530 0x7f764410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.326+0000 7f764b351640 1 --2- 172.21.15.129:0/3682627420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644103b70 0x7f7644103ff0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.326+0000 7f764b351640 1 --2- 172.21.15.129:0/3682627420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7644102980 0x7f7644102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.326+0000 7f764b351640 1 -- 172.21.15.129:0/3682627420 >> 172.21.15.129:0/3682627420 conn(0x7f76440fdf10 msgr2=0x7f7644100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.326+0000 7f764b351640 1 -- 172.21.15.129:0/3682627420 shutdown_connections 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.326+0000 7f764b351640 1 -- 172.21.15.129:0/3682627420 wait complete. 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f764b351640 1 Processor -- start 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f764b351640 1 -- start start 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f764b351640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644102980 0x7f7644198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f764b351640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7644103b70 0x7f7644199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f764b351640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7644104530 0x7f76441a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f764b351640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f764410e490 con 0x7f7644102980 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f764b351640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f764410e310 con 0x7f7644103b70 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f764b351640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f764410e610 con 0x7f7644104530 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76490ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644102980 0x7f7644198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76490ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644102980 0x7f7644198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51056/0 (socket says 172.21.15.129:51056) 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76490ed640 1 -- 172.21.15.129:0/1411564042 learned_addr learned my addr 172.21.15.129:0/1411564042 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76488ec640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7644103b70 0x7f7644199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76498ee640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7644104530 0x7f76441a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76490ed640 1 -- 172.21.15.129:0/1411564042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7644104530 msgr2=0x7f76441a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76490ed640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7644104530 0x7f76441a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76490ed640 1 -- 172.21.15.129:0/1411564042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7644103b70 msgr2=0x7f7644199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76490ed640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7644103b70 0x7f7644199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76490ed640 1 -- 172.21.15.129:0/1411564042 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76441a0810 con 0x7f7644102980 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.327+0000 7f76490ed640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644102980 0x7f7644198bf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f764000d950 tx=0x7f764000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.328+0000 7f763a7fc640 1 -- 172.21.15.129:0/1411564042 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640017020 con 0x7f7644102980 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.328+0000 7f763a7fc640 1 -- 172.21.15.129:0/1411564042 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7640009c30 con 0x7f7644102980 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.328+0000 7f764b351640 1 -- 172.21.15.129:0/1411564042 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76441a0b00 con 0x7f7644102980 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.328+0000 7f763a7fc640 1 -- 172.21.15.129:0/1411564042 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764000f6b0 con 0x7f7644102980 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.328+0000 7f764b351640 1 -- 172.21.15.129:0/1411564042 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76441a1020 con 0x7f7644102980 2024-04-26T21:18:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.330+0000 7f763a7fc640 1 -- 172.21.15.129:0/1411564042 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f764001a020 con 0x7f7644102980 2024-04-26T21:18:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.330+0000 7f764e06d740 1 Processor -- start 2024-04-26T21:18:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f764e06d740 1 -- start start 2024-04-26T21:18:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f764e06d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1acc0 0x555afff1b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f764e06d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1b6b0 0x555afff201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f764e06d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff20820 0x555afff22cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f764e06d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555affe7bb00 con 0x555afff1acc0 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f764e06d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555affe17920 con 0x555afff1b6b0 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f764e06d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555affe77ac0 con 0x555afff20820 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f763a7fc640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f76300a3c40 0x7f76300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1acc0 0x555afff1b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76490ed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1b6b0 0x555afff201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1acc0 0x555afff1b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51060/0 (socket says 172.21.15.129:51060) 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 -- 172.21.15.129:0/3071822084 learned_addr learned my addr 172.21.15.129:0/3071822084 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f76300a3c40 0x7f76300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76488ec640 1 --2- 172.21.15.129:0/3071822084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff20820 0x555afff22cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f763a7fc640 1 -- 172.21.15.129:0/1411564042 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f76400d0210 con 0x7f7644102980 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 -- 172.21.15.129:0/3071822084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff20820 msgr2=0x555afff22cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 --2- 172.21.15.129:0/3071822084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff20820 0x555afff22cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 -- 172.21.15.129:0/3071822084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1b6b0 msgr2=0x555afff201b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 --2- 172.21.15.129:0/3071822084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1b6b0 0x555afff201b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 -- 172.21.15.129:0/3071822084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555affc557e0 con 0x555afff1acc0 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.331+0000 7f76498ee640 1 --2- 172.21.15.129:0/3071822084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1acc0 0x555afff1b0e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f763c014eb0 tx=0x7f763c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193c9428cb542a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f76498ee640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f76300a3c40 0x7f76300a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f763c011cb0 tx=0x7f763c016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f762f7fe640 1 -- 172.21.15.129:0/3071822084 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c009c20 con 0x555afff1acc0 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f762f7fe640 1 -- 172.21.15.129:0/3071822084 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c014810 con 0x555afff1acc0 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f762f7fe640 1 -- 172.21.15.129:0/3071822084 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c003bd0 con 0x555afff1acc0 2024-04-26T21:18:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f764e06d740 1 -- 172.21.15.129:0/3071822084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1acc0 msgr2=0x555afff1b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f764e06d740 1 --2- 172.21.15.129:0/3071822084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1acc0 0x555afff1b0e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f763c014eb0 tx=0x7f763c0117b0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f764e06d740 1 -- 172.21.15.129:0/3071822084 shutdown_connections 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f764e06d740 1 --2- 172.21.15.129:0/3071822084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1b6b0 0x555afff201b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f764e06d740 1 --2- 172.21.15.129:0/3071822084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff20820 0x555afff22cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f764e06d740 1 --2- 172.21.15.129:0/3071822084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1acc0 0x555afff1b0e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f764e06d740 1 -- 172.21.15.129:0/3071822084 >> 172.21.15.129:0/3071822084 conn(0x555afff11f20 msgr2=0x555afff12320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f764e06d740 1 -- 172.21.15.129:0/3071822084 shutdown_connections 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.332+0000 7f764e06d740 1 -- 172.21.15.129:0/3071822084 wait complete. 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f764e06d740 1 -- start start 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f764e06d740 1 Processor -- start 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f764e06d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f764e06d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 0x555afff25610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f764e06d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f764e06d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555afff23770 con 0x555afff20820 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f764e06d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555affe155d0 con 0x555afff1acc0 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f764e06d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555afff238f0 con 0x555afff1b6b0 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f764e06d740 10 client.0 _ll_register_callbacks cb 0x555affe03080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f76498ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f76498ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51068/0 (socket says 172.21.15.129:51068) 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f76498ee640 1 -- 172.21.15.129:0/3042883985 learned_addr learned my addr 172.21.15.129:0/3042883985 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.333+0000 7f76488ec640 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 0x555afff25610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f76490ed640 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f76498ee640 1 -- 172.21.15.129:0/3042883985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 msgr2=0x555afff25610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f76498ee640 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 0x555afff25610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f76498ee640 1 -- 172.21.15.129:0/3042883985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 msgr2=0x555afff250d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f76498ee640 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff250d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f76498ee640 1 -- 172.21.15.129:0/3042883985 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555affcb5f70 con 0x555afff20820 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f76490ed640 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f76498ee640 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2c650 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f763c014bf0 tx=0x7f763c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55feb1cbb09c27c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f762cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f762cff9640 1 -- 172.21.15.129:0/3042883985 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c003b80 con 0x555afff20820 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f762cff9640 1 -- 172.21.15.129:0/3042883985 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f763c013460 con 0x555afff20820 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f764e06d740 1 -- 172.21.15.129:0/3042883985 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c014810 con 0x555afff20820 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f762cff9640 1 -- 172.21.15.129:0/3042883985 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f763c013660 con 0x555afff20820 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f764e06d740 10 client.15432 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f764e06d740 1 -- 172.21.15.129:0/3042883985 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555affb4d170 con 0x555afff20820 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f762cff9640 1 -- 172.21.15.129:0/3042883985 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c014810 con 0x555afff20820 2024-04-26T21:18:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.334+0000 7f762cff9640 1 -- 172.21.15.129:0/3042883985 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f763c01d070 con 0x555afff20820 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.335+0000 7f764e06d740 10 client.15432 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.335+0000 7f764e06d740 10 client.15432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.335+0000 7f764e06d740 20 client.15432 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.335+0000 7f764e06d740 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555afff30a30 0x555afff32ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.335+0000 7f764e06d740 4 client.15432 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.335+0000 7f764e06d740 1 -- 172.21.15.129:0/3042883985 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555affe89710 con 0x555afff30a30 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.335+0000 7f76490ed640 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555afff30a30 0x555afff32ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.335+0000 7f762cff9640 10 client.15432 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.335+0000 7f76490ed640 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555afff30a30 0x555afff32ed0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b49a8e3d4b83e05 server_cookie=bf6060312317f44e in_seq=0 out_seq=0 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f762cff9640 1 -- 172.21.15.129:0/3042883985 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555affe89710 con 0x555afff30a30 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f762cff9640 10 client.15432 handle_command_reply: tid=0 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 client.15432 shutdown 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 -- 172.21.15.129:0/3042883985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 msgr2=0x555afff2c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2c650 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f763c014bf0 tx=0x7f763c009240 comp rx=0 tx=0).stop 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 -- 172.21.15.129:0/3042883985 shutdown_connections 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555afff30a30 0x555afff32ed0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2c650 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 0x555afff25610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f762cff9640 0 client.15432 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 --2- 172.21.15.129:0/3042883985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 -- 172.21.15.129:0/3042883985 >> 172.21.15.129:0/3042883985 conn(0x555afff11f20 msgr2=0x555afff110e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 -- 172.21.15.129:0/3042883985 shutdown_connections 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 1 -- 172.21.15.129:0/3042883985 wait complete. 2024-04-26T21:18:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.336+0000 7f764e06d740 20 client.15432 trim_cache size 0 max 16384 2024-04-26T21:18:42.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f764e06d740 1 Processor -- start 2024-04-26T21:18:42.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f764e06d740 1 -- start start 2024-04-26T21:18:42.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f764e06d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1acc0 0x555afff14c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f764e06d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1b6b0 0x555afff2c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f764e06d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff20820 0x555afff15140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f764e06d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76441a5bb0 con 0x555afff1b6b0 2024-04-26T21:18:42.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f764e06d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76441a8430 con 0x555afff20820 2024-04-26T21:18:42.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f764e06d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f76441a6220 con 0x555afff1acc0 2024-04-26T21:18:42.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f76488ec640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1b6b0 0x555afff2c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f76488ec640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1b6b0 0x555afff2c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51080/0 (socket says 172.21.15.129:51080) 2024-04-26T21:18:42.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f76488ec640 1 -- 172.21.15.129:0/1189602487 learned_addr learned my addr 172.21.15.129:0/1189602487 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f76488ec640 1 -- 172.21.15.129:0/1189602487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1acc0 msgr2=0x555afff14c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f76488ec640 1 --2- 172.21.15.129:0/1189602487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1acc0 0x555afff14c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f76488ec640 1 -- 172.21.15.129:0/1189602487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff20820 msgr2=0x555afff15140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f76488ec640 1 --2- 172.21.15.129:0/1189602487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff20820 0x555afff15140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.354+0000 7f76488ec640 1 -- 172.21.15.129:0/1189602487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555afff11d50 con 0x555afff1b6b0 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f76488ec640 1 --2- 172.21.15.129:0/1189602487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1b6b0 0x555afff2c610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7634033000 tx=0x7f7634031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85687c8d07f27c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f762effd640 1 -- 172.21.15.129:0/1189602487 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634035040 con 0x555afff1b6b0 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f762effd640 1 -- 172.21.15.129:0/1189602487 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7634004030 con 0x555afff1b6b0 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f762effd640 1 -- 172.21.15.129:0/1189602487 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634003c50 con 0x555afff1b6b0 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f764e06d740 1 -- 172.21.15.129:0/1189602487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1b6b0 msgr2=0x555afff2c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f764e06d740 1 --2- 172.21.15.129:0/1189602487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1b6b0 0x555afff2c610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7634033000 tx=0x7f7634031de0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f764e06d740 1 -- 172.21.15.129:0/1189602487 shutdown_connections 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f764e06d740 1 --2- 172.21.15.129:0/1189602487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff20820 0x555afff15140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f764e06d740 1 --2- 172.21.15.129:0/1189602487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff1b6b0 0x555afff2c610 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f764e06d740 1 --2- 172.21.15.129:0/1189602487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1acc0 0x555afff14c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f764e06d740 1 -- 172.21.15.129:0/1189602487 >> 172.21.15.129:0/1189602487 conn(0x555afff11f20 msgr2=0x555afff1cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f764e06d740 1 -- 172.21.15.129:0/1189602487 shutdown_connections 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.355+0000 7f764e06d740 1 -- 172.21.15.129:0/1189602487 wait complete. 2024-04-26T21:18:42.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f764e06d740 1 -- start start 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f764e06d740 1 Processor -- start 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f764e06d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f764e06d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 0x555afff2a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f764e06d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f764e06d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76441a68f0 con 0x555afff20820 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f764e06d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76441a7d60 con 0x555afff1acc0 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f764e06d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f76441a6fc0 con 0x555afff1b6b0 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f764e06d740 10 client.0 _ll_register_callbacks cb 0x7f763c014cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76498ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76498ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51088/0 (socket says 172.21.15.129:51088) 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76498ee640 1 -- 172.21.15.129:0/347869336 learned_addr learned my addr 172.21.15.129:0/347869336 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76488ec640 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 0x555afff2a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76490ed640 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76498ee640 1 -- 172.21.15.129:0/347869336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 msgr2=0x555afff2a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76498ee640 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 0x555afff2a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76498ee640 1 -- 172.21.15.129:0/347869336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 msgr2=0x555afff29bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76498ee640 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff29bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76498ee640 1 -- 172.21.15.129:0/347869336 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555afff2b6e0 con 0x555afff20820 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.356+0000 7f76490ed640 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff29bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f76498ee640 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2b0f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f763c00a010 tx=0x7f763c014de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29ba8356815166d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f762cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f762cff9640 1 -- 172.21.15.129:0/347869336 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c009c20 con 0x555afff20820 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f764e06d740 1 -- 172.21.15.129:0/347869336 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7634004030 con 0x555afff20820 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f762cff9640 1 -- 172.21.15.129:0/347869336 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f763c025040 con 0x555afff20820 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f762cff9640 1 -- 172.21.15.129:0/347869336 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f763c04c050 con 0x555afff20820 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f764e06d740 10 client.15438 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f764e06d740 1 -- 172.21.15.129:0/347869336 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555afff97010 con 0x555afff20820 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f762cff9640 1 -- 172.21.15.129:0/347869336 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7634004030 con 0x555afff20820 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f762cff9640 1 -- 172.21.15.129:0/347869336 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f763c01d040 con 0x555afff20820 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f764e06d740 10 client.15438 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f764e06d740 10 client.15438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f764e06d740 20 client.15438 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f764e06d740 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555afff978f0 0x555afff99d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f764e06d740 4 client.15438 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.357+0000 7f764e06d740 1 -- 172.21.15.129:0/347869336 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555afff24ec0 con 0x555afff978f0 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.358+0000 7f76490ed640 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555afff978f0 0x555afff99d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.358+0000 7f76490ed640 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555afff978f0 0x555afff99d10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76958278c8ffb2c server_cookie=a02ee5b54d1cb6d1 in_seq=0 out_seq=0 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.358+0000 7f762cff9640 10 client.15438 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f762cff9640 1 -- 172.21.15.129:0/347869336 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555afff24ec0 con 0x555afff978f0 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f762cff9640 10 client.15438 handle_command_reply: tid=0 2024-04-26T21:18:42.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f764e06d740 1 client.15438 shutdown 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f764e06d740 1 -- 172.21.15.129:0/347869336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 msgr2=0x555afff2b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f764e06d740 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2b0f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f763c00a010 tx=0x7f763c014de0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f764e06d740 1 -- 172.21.15.129:0/347869336 shutdown_connections 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f764e06d740 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555afff978f0 0x555afff99d10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f762cff9640 0 client.15438 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f764e06d740 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555afff20820 0x555afff2b0f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f764e06d740 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555afff1b6b0 0x555afff2a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f764e06d740 1 --2- 172.21.15.129:0/347869336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555afff1acc0 0x555afff29bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.359+0000 7f764e06d740 1 -- 172.21.15.129:0/347869336 >> 172.21.15.129:0/347869336 conn(0x555afff11f20 msgr2=0x555afff10850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764e06d740 1 -- 172.21.15.129:0/347869336 shutdown_connections 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764e06d740 1 -- 172.21.15.129:0/347869336 wait complete. 2024-04-26T21:18:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764e06d740 20 client.15438 trim_cache size 0 max 16384 2024-04-26T21:18:42.368 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:18:42.368 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:18:42.368 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "be475b04-1ce9-48f5-b314-2f7ffc61ee3e", 2024-04-26T21:18:42.368 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:18:42.368 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:18:42.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 -- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f76300a3c40 msgr2=0x7f76300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f76300a3c40 0x7f76300a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f763c011cb0 tx=0x7f763c016000 comp rx=0 tx=0).stop 2024-04-26T21:18:42.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 -- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644102980 msgr2=0x7f7644198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644102980 0x7f7644198bf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f764000d950 tx=0x7f764000de10 comp rx=0 tx=0).stop 2024-04-26T21:18:42.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 -- 172.21.15.129:0/1411564042 shutdown_connections 2024-04-26T21:18:42.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f76300a3c40 0x7f76300a6100 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7644103b70 0x7f7644199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7644104530 0x7f76441a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 --2- 172.21.15.129:0/1411564042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7644102980 0x7f7644198bf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 -- 172.21.15.129:0/1411564042 >> 172.21.15.129:0/1411564042 conn(0x7f76440fdf10 msgr2=0x7f7644106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 -- 172.21.15.129:0/1411564042 shutdown_connections 2024-04-26T21:18:42.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.360+0000 7f764b351640 1 -- 172.21.15.129:0/1411564042 wait complete. 2024-04-26T21:18:42.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:18:42.372 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:18:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.471+0000 7f3abdd39640 1 -- 172.21.15.129:0/3062892957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 msgr2=0x7f3ab81032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.471+0000 7f3abdd39640 1 --2- 172.21.15.129:0/3062892957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 0x7f3ab81032c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009900 tx=0x7f3aa802a410 comp rx=0 tx=0).stop 2024-04-26T21:18:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.471+0000 7f3abdd39640 1 -- 172.21.15.129:0/3062892957 shutdown_connections 2024-04-26T21:18:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.471+0000 7f3abdd39640 1 --2- 172.21.15.129:0/3062892957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab8101af0 0x7f3ab8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.471+0000 7f3abdd39640 1 --2- 172.21.15.129:0/3062892957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ab81024b0 0x7f3ab8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.471+0000 7f3abdd39640 1 --2- 172.21.15.129:0/3062892957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 0x7f3ab81032c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.471+0000 7f3abdd39640 1 -- 172.21.15.129:0/3062892957 >> 172.21.15.129:0/3062892957 conn(0x7f3ab80fc560 msgr2=0x7f3ab80fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.471+0000 7f3abdd39640 1 -- 172.21.15.129:0/3062892957 shutdown_connections 2024-04-26T21:18:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.471+0000 7f3abdd39640 1 -- 172.21.15.129:0/3062892957 wait complete. 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3abdd39640 1 Processor -- start 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3abdd39640 1 -- start start 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3abdd39640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ab8101af0 0x7f3ab81015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3abdd39640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab81024b0 0x7f3ab819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3abdd39640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 0x7f3ab819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3abdd39640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810d3b0 con 0x7f3ab8102ea0 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3abdd39640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810d230 con 0x7f3ab81024b0 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3abdd39640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810d530 con 0x7f3ab8101af0 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 0x7f3ab819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 0x7f3ab819fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51108/0 (socket says 172.21.15.129:51108) 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3ab7fff640 1 -- 172.21.15.129:0/2912131904 learned_addr learned my addr 172.21.15.129:0/2912131904 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab81024b0 0x7f3ab819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.472+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ab8101af0 0x7f3ab81015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3ab7fff640 1 -- 172.21.15.129:0/2912131904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ab8101af0 msgr2=0x7f3ab81015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ab8101af0 0x7f3ab81015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3ab7fff640 1 -- 172.21.15.129:0/2912131904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab81024b0 msgr2=0x7f3ab819d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab81024b0 0x7f3ab819d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3ab7fff640 1 -- 172.21.15.129:0/2912131904 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab81a07a0 con 0x7f3ab8102ea0 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 0x7f3ab819fee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000cce0 tx=0x7f3ab000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3ab4ff9640 1 -- 172.21.15.129:0/2912131904 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab000f450 con 0x7f3ab8102ea0 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3ab4ff9640 1 -- 172.21.15.129:0/2912131904 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab000faa0 con 0x7f3ab8102ea0 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3abdd39640 1 -- 172.21.15.129:0/2912131904 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81a0a90 con 0x7f3ab8102ea0 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3ab4ff9640 1 -- 172.21.15.129:0/2912131904 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0018b70 con 0x7f3ab8102ea0 2024-04-26T21:18:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.473+0000 7f3abdd39640 1 -- 172.21.15.129:0/2912131904 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a0f90 con 0x7f3ab8102ea0 2024-04-26T21:18:42.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.475+0000 7f3ab4ff9640 1 -- 172.21.15.129:0/2912131904 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3ab0018d10 con 0x7f3ab8102ea0 2024-04-26T21:18:42.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.475+0000 7f3ac0a4f740 1 Processor -- start 2024-04-26T21:18:42.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.475+0000 7f3ac0a4f740 1 -- start start 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.475+0000 7f3ab4ff9640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3aa00a3c40 0x7f3aa00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 0x56238f288300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f2890b0 0x56238f289530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f289a70 0x56238f290300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56238f1ecd80 con 0x56238f2890b0 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56238f1a68b0 con 0x56238f287ee0 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56238f1e8d40 con 0x56238f289a70 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3aa00a3c40 0x7f3aa00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab4ff9640 1 -- 172.21.15.129:0/2912131904 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3ab00d0a20 con 0x7f3ab8102ea0 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f2890b0 0x56238f289530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f2890b0 0x56238f289530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51112/0 (socket says 172.21.15.129:51112) 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 -- 172.21.15.129:0/724416028 learned_addr learned my addr 172.21.15.129:0/724416028 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/724416028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 0x56238f288300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/724416028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f289a70 0x56238f290300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 -- 172.21.15.129:0/724416028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f289a70 msgr2=0x56238f290300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/724416028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f289a70 0x56238f290300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 -- 172.21.15.129:0/724416028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 msgr2=0x56238f288300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/724416028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 0x56238f288300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 -- 172.21.15.129:0/724416028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238f177a40 con 0x56238f2890b0 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3aa00a3c40 0x7f3aa00a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0119b0 tx=0x7f3aac009be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/724416028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f2890b0 0x56238f289530 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00f790 tx=0x7f3aac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76efb3970a35466 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3a9dffb640 1 -- 172.21.15.129:0/724416028 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00fe50 con 0x56238f2890b0 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3a9dffb640 1 -- 172.21.15.129:0/724416028 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac011020 con 0x56238f2890b0 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.476+0000 7f3a9dffb640 1 -- 172.21.15.129:0/724416028 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac020d00 con 0x56238f2890b0 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/724416028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f2890b0 msgr2=0x56238f289530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/724416028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f2890b0 0x56238f289530 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00f790 tx=0x7f3aac01d000 comp rx=0 tx=0).stop 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/724416028 shutdown_connections 2024-04-26T21:18:42.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/724416028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f289a70 0x56238f290300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/724416028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f2890b0 0x56238f289530 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/724416028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 0x56238f288300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/724416028 >> 172.21.15.129:0/724416028 conn(0x56238f2833d0 msgr2=0x56238f2837d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/724416028 shutdown_connections 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/724416028 wait complete. 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.477+0000 7f3ac0a4f740 1 -- start start 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ac0a4f740 1 Processor -- start 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f287ee0 0x56238f292320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 0x56238f292860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f289a70 0x56238f2998a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56238f290a20 con 0x56238f287ee0 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56238f173d10 con 0x56238f289a70 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56238f290ba0 con 0x56238f2890b0 2024-04-26T21:18:42.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ac0a4f740 10 client.0 _ll_register_callbacks cb 0x56238f1a0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f287ee0 0x56238f292320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f287ee0 0x56238f292320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51122/0 (socket says 172.21.15.129:51122) 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab77fe640 1 -- 172.21.15.129:0/1565463116 learned_addr learned my addr 172.21.15.129:0/1565463116 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 0x56238f292860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f289a70 0x56238f2998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab77fe640 1 -- 172.21.15.129:0/1565463116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 msgr2=0x56238f292860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 0x56238f292860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab77fe640 1 -- 172.21.15.129:0/1565463116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f289a70 msgr2=0x56238f2998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f289a70 0x56238f2998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab77fe640 1 -- 172.21.15.129:0/1565463116 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238efc87d0 con 0x56238f287ee0 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f287ee0 0x56238f292320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009900 tx=0x7f3aa8032cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42d18b228b0060d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3a96ffd640 1 -- 172.21.15.129:0/1565463116 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8035040 con 0x56238f287ee0 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.478+0000 7f3a96ffd640 1 -- 172.21.15.129:0/1565463116 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3aa8004240 con 0x56238f287ee0 2024-04-26T21:18:42.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1565463116 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac011020 con 0x56238f287ee0 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3a96ffd640 1 -- 172.21.15.129:0/1565463116 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3aa8004440 con 0x56238f287ee0 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ac0a4f740 10 client.15450 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1565463116 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56238eefe3f0 con 0x56238f287ee0 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3a96ffd640 1 -- 172.21.15.129:0/1565463116 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac011020 con 0x56238f287ee0 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3a96ffd640 1 -- 172.21.15.129:0/1565463116 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f3aa8004bb0 con 0x56238f287ee0 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ac0a4f740 10 client.15450 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ac0a4f740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ac0a4f740 20 client.15450 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56238f29ddf0 0x56238f2a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ac0a4f740 4 client.15450 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1565463116 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56238f1fa990 con 0x56238f29ddf0 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.479+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56238f29ddf0 0x56238f2a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.480+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56238f29ddf0 0x56238f2a0270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da056287c71fe06 server_cookie=9a610d72c9f22624 in_seq=0 out_seq=0 2024-04-26T21:18:42.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.480+0000 7f3a96ffd640 10 client.15450 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.480+0000 7f3a96ffd640 1 -- 172.21.15.129:0/1565463116 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56238f1fa990 con 0x56238f29ddf0 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.480+0000 7f3a96ffd640 10 client.15450 handle_command_reply: tid=0 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.480+0000 7f3ac0a4f740 1 client.15450 shutdown 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1565463116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f287ee0 msgr2=0x56238f292320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f287ee0 0x56238f292320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009900 tx=0x7f3aa8032cf0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1565463116 shutdown_connections 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56238f29ddf0 0x56238f2a0270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f289a70 0x56238f2998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 0x56238f292860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3a96ffd640 0 client.15450 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1565463116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f287ee0 0x56238f292320 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1565463116 >> 172.21.15.129:0/1565463116 conn(0x56238f2833d0 msgr2=0x56238f282550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1565463116 shutdown_connections 2024-04-26T21:18:42.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1565463116 wait complete. 2024-04-26T21:18:42.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.481+0000 7f3ac0a4f740 20 client.15450 trim_cache size 0 max 16384 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ac0a4f740 1 Processor -- start 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ac0a4f740 1 -- start start 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f287ee0 0x56238f2867c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f2890b0 0x56238f286d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f2a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81a7c90 con 0x56238f289a70 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81aa510 con 0x56238f2890b0 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81a8300 con 0x56238f287ee0 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f2a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f2a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51136/0 (socket says 172.21.15.129:51136) 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab7fff640 1 -- 172.21.15.129:0/1322825752 learned_addr learned my addr 172.21.15.129:0/1322825752 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/1322825752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f287ee0 0x56238f2867c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab7fff640 1 -- 172.21.15.129:0/1322825752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f287ee0 msgr2=0x56238f2867c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/1322825752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f287ee0 0x56238f2867c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab7fff640 1 -- 172.21.15.129:0/1322825752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f2890b0 msgr2=0x56238f286d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/1322825752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f2890b0 0x56238f286d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab7fff640 1 -- 172.21.15.129:0/1322825752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238eebe0f0 con 0x56238f289a70 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.498+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/1322825752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f287ee0 0x56238f2867c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/1322825752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f2a4b30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3ab001f000 tx=0x7f3ab001ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee63c37975e17449 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3a9d7fa640 1 -- 172.21.15.129:0/1322825752 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab009cb90 con 0x56238f289a70 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3a9d7fa640 1 -- 172.21.15.129:0/1322825752 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab00a0f40 con 0x56238f289a70 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3a9d7fa640 1 -- 172.21.15.129:0/1322825752 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00bd070 con 0x56238f289a70 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1322825752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 msgr2=0x56238f2a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1322825752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f2a4b30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3ab001f000 tx=0x7f3ab001ff40 comp rx=0 tx=0).stop 2024-04-26T21:18:42.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1322825752 shutdown_connections 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1322825752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f2890b0 0x56238f286d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1322825752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f2a4b30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/1322825752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f287ee0 0x56238f2867c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1322825752 >> 172.21.15.129:0/1322825752 conn(0x56238f2833d0 msgr2=0x56238f2860b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1322825752 shutdown_connections 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.499+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/1322825752 wait complete. 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ac0a4f740 1 -- start start 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ac0a4f740 1 Processor -- start 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 0x56238f2ff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 0x56238f2ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ac0a4f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f306be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81a89d0 con 0x56238f289a70 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81a9e40 con 0x56238f287ee0 2024-04-26T21:18:42.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ac0a4f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81a90a0 con 0x56238f2890b0 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ac0a4f740 10 client.0 _ll_register_callbacks cb 0x7f3aa8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f306be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f306be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55430/0 (socket says 172.21.15.129:55430) 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ab7fff640 1 -- 172.21.15.129:0/3848809955 learned_addr learned my addr 172.21.15.129:0/3848809955 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ab6ffd640 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 0x56238f2ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 0x56238f2ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ab7fff640 1 -- 172.21.15.129:0/3848809955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 msgr2=0x56238f2ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 0x56238f2ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ab7fff640 1 -- 172.21.15.129:0/3848809955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 msgr2=0x56238f2ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.500+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 0x56238f2ff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ab7fff640 1 -- 172.21.15.129:0/3848809955 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238f1ad4b0 con 0x56238f289a70 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ab7fff640 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f306be0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3ab009c890 tx=0x7f3ab00a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f97781ea440f805 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3a96ffd640 1 -- 172.21.15.129:0/3848809955 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00a5800 con 0x56238f289a70 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3a96ffd640 1 -- 172.21.15.129:0/3848809955 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3ab009ae90 con 0x56238f289a70 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/3848809955 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab00a0f40 con 0x56238f289a70 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3a96ffd640 1 -- 172.21.15.129:0/3848809955 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3ab00bd0f0 con 0x56238f289a70 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ac0a4f740 10 client.15456 fetch_fsmap learned FSMap version 19 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/3848809955 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56238f308280 con 0x56238f289a70 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3a96ffd640 1 -- 172.21.15.129:0/3848809955 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab00a0f40 con 0x56238f289a70 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3a96ffd640 1 -- 172.21.15.129:0/3848809955 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f3ab00988e0 con 0x56238f289a70 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ac0a4f740 10 client.15456 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ac0a4f740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ac0a4f740 20 client.15456 populate_metadata read hostname 'smithi129' 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56238f308bc0 0x56238f30afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ac0a4f740 4 client.15456 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:18:42.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/3848809955 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56238f1f4800 con 0x56238f308bc0 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.501+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56238f308bc0 0x56238f30afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3a96ffd640 10 client.15456 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ab77fe640 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56238f308bc0 0x56238f30afe0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a81f246e88c236e server_cookie=55abd8c2b399cfd8 in_seq=0 out_seq=0 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3a96ffd640 1 -- 172.21.15.129:0/3848809955 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56238f1f4800 con 0x56238f308bc0 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3a96ffd640 10 client.15456 handle_command_reply: tid=0 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 client.15456 shutdown 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/3848809955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 msgr2=0x56238f306be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f306be0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3ab009c890 tx=0x7f3ab00a5710 comp rx=0 tx=0).stop 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/3848809955 shutdown_connections 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56238f308bc0 0x56238f30afe0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56238f289a70 0x56238f306be0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56238f2890b0 0x56238f2ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3a96ffd640 0 client.15456 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 --2- 172.21.15.129:0/3848809955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238f287ee0 0x56238f2ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/3848809955 >> 172.21.15.129:0/3848809955 conn(0x56238f2833d0 msgr2=0x56238f286940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/3848809955 shutdown_connections 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 1 -- 172.21.15.129:0/3848809955 wait complete. 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.502+0000 7f3ac0a4f740 20 client.15456 trim_cache size 0 max 16384 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stdout: "be475b04-1ce9-48f5-b314-2f7ffc61ee3e": { 2024-04-26T21:18:42.511 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:18:42.512 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "be475b04-1ce9-48f5-b314-2f7ffc61ee3e", 2024-04-26T21:18:42.512 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:18:42.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:18:42.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:18:42.512 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 -- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3aa00a3c40 msgr2=0x7f3aa00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3aa00a3c40 0x7f3aa00a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0119b0 tx=0x7f3aac009be0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 -- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 msgr2=0x7f3ab819fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 0x7f3ab819fee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000cce0 tx=0x7f3ab000a620 comp rx=0 tx=0).stop 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 -- 172.21.15.129:0/2912131904 shutdown_connections 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3aa00a3c40 0x7f3aa00a6100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab81024b0 0x7f3ab819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ab8102ea0 0x7f3ab819fee0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 --2- 172.21.15.129:0/2912131904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ab8101af0 0x7f3ab81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 -- 172.21.15.129:0/2912131904 >> 172.21.15.129:0/2912131904 conn(0x7f3ab80fc560 msgr2=0x7f3ab8105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 -- 172.21.15.129:0/2912131904 shutdown_connections 2024-04-26T21:18:42.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:18:42.503+0000 7f3abdd39640 1 -- 172.21.15.129:0/2912131904 wait complete. 2024-04-26T21:18:42.514 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:42 smithi129 ceph-mon[29237]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:42.515 INFO:tasks.cephfs.filesystem:scrub status for tag:be475b04-1ce9-48f5-b314-2f7ffc61ee3e - {'path': '/', 'tag': 'be475b04-1ce9-48f5-b314-2f7ffc61ee3e', 'options': 'recursive,force'} 2024-04-26T21:18:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:42 smithi171 ceph-mon[33478]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:42.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:42 smithi150 ceph-mon[32290]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:43 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:18:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:43 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:43 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:18:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:43 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:43 smithi129 ceph-mon[29237]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-26T21:18:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:43 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:18:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:43 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:43 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:18:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:43 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:43 smithi171 ceph-mon[33478]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-26T21:18:43.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:43 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:18:43.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:43 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:43.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:43 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:18:43.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:43 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:18:43.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:43 smithi150 ceph-mon[32290]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-26T21:18:44.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:44 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:18:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:44 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:18:44.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:44 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:18:45.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:45 smithi129 ceph-mon[29237]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 597 B/s wr, 1 op/s 2024-04-26T21:18:45.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:45 smithi171 ceph-mon[33478]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 597 B/s wr, 1 op/s 2024-04-26T21:18:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:45 smithi150 ceph-mon[32290]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 597 B/s wr, 1 op/s 2024-04-26T21:18:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:48 smithi150 ceph-mon[32290]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 682 B/s wr, 2 op/s 2024-04-26T21:18:48.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:48 smithi129 ceph-mon[29237]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 682 B/s wr, 2 op/s 2024-04-26T21:18:48.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:48 smithi171 ceph-mon[33478]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 682 B/s wr, 2 op/s 2024-04-26T21:18:50.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:50 smithi150 ceph-mon[32290]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-26T21:18:50.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:50 smithi129 ceph-mon[29237]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-26T21:18:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:50 smithi171 ceph-mon[33478]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-26T21:18:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:52 smithi150 ceph-mon[32290]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-26T21:18:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:52 smithi129 ceph-mon[29237]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-26T21:18:52.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:52 smithi171 ceph-mon[33478]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-26T21:18:54.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:54 smithi150 ceph-mon[32290]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-26T21:18:54.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:54 smithi129 ceph-mon[29237]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-26T21:18:54.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:54 smithi171 ceph-mon[33478]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-26T21:18:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:56 smithi129 ceph-mon[29237]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:18:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:56 smithi171 ceph-mon[33478]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:18:56.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:56 smithi150 ceph-mon[32290]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:18:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:18:58 smithi129 ceph-mon[29237]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:18:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:18:58 smithi171 ceph-mon[33478]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:18:58.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:18:58 smithi150 ceph-mon[32290]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:19:00.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:00 smithi129 ceph-mon[29237]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-04-26T21:19:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:00 smithi171 ceph-mon[33478]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-04-26T21:19:00.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:00 smithi150 ceph-mon[32290]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-04-26T21:19:02.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:02 smithi129 ceph-mon[29237]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:02.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:02 smithi171 ceph-mon[33478]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:02.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:02 smithi150 ceph-mon[32290]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:04 smithi129 ceph-mon[29237]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:04.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:04 smithi171 ceph-mon[33478]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:04.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:04 smithi150 ceph-mon[32290]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:05.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:05 smithi129 ceph-mon[29237]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:05.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:05 smithi171 ceph-mon[33478]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:05.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:05 smithi150 ceph-mon[32290]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:08.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:08 smithi150 ceph-mon[32290]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:08.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:08 smithi129 ceph-mon[29237]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:08.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:08 smithi171 ceph-mon[33478]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:10 smithi150 ceph-mon[32290]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:10 smithi129 ceph-mon[29237]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:10 smithi171 ceph-mon[33478]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:12.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:19:12.516 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:19:12.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.627+0000 7f1cb5874640 1 -- 172.21.15.129:0/1043092293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0108650 msgr2=0x7f1cb010aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.627+0000 7f1cb5874640 1 --2- 172.21.15.129:0/1043092293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0108650 0x7f1cb010aaa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009ca0 tx=0x7f1ca802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.628+0000 7f1cb5874640 1 -- 172.21.15.129:0/1043092293 shutdown_connections 2024-04-26T21:19:12.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.628+0000 7f1cb5874640 1 --2- 172.21.15.129:0/1043092293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0108650 0x7f1cb010aaa0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.628+0000 7f1cb5874640 1 --2- 172.21.15.129:0/1043092293 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb0105d20 0x7f1cb0108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.628+0000 7f1cb5874640 1 --2- 172.21.15.129:0/1043092293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb01033f0 0x7f1cb01057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.628+0000 7f1cb5874640 1 -- 172.21.15.129:0/1043092293 >> 172.21.15.129:0/1043092293 conn(0x7f1cb00fcde0 msgr2=0x7f1cb00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.628+0000 7f1cb5874640 1 -- 172.21.15.129:0/1043092293 shutdown_connections 2024-04-26T21:19:12.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.628+0000 7f1cb5874640 1 -- 172.21.15.129:0/1043092293 wait complete. 2024-04-26T21:19:12.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.628+0000 7f1cb5874640 1 Processor -- start 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cb5874640 1 -- start start 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cb5874640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb01033f0 0x7f1cb0198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cb5874640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0105d20 0x7f1cb0199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cb5874640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb0108650 0x7f1cb01a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cb5874640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb010e4c0 con 0x7f1cb0105d20 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cb5874640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb010e340 con 0x7f1cb01033f0 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cae7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0105d20 0x7f1cb0199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cb5874640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb010e640 con 0x7f1cb0108650 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cae7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0105d20 0x7f1cb0199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58040/0 (socket says 172.21.15.129:58040) 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cae7fc640 1 -- 172.21.15.129:0/3431225903 learned_addr learned my addr 172.21.15.129:0/3431225903 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1caeffd640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb01033f0 0x7f1cb0198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1caf7fe640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb0108650 0x7f1cb01a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cae7fc640 1 -- 172.21.15.129:0/3431225903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb0108650 msgr2=0x7f1cb01a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb0108650 0x7f1cb01a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cae7fc640 1 -- 172.21.15.129:0/3431225903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb01033f0 msgr2=0x7f1cb0198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb01033f0 0x7f1cb0198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1cae7fc640 1 -- 172.21.15.129:0/3431225903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb01a07e0 con 0x7f1cb0105d20 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.629+0000 7f1caf7fe640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb0108650 0x7f1cb01a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.630+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0105d20 0x7f1cb0199100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400d740 tx=0x7f1ca400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.630+0000 7f1cb4872640 1 -- 172.21.15.129:0/3431225903 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4017070 con 0x7f1cb0105d20 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.630+0000 7f1cb5874640 1 -- 172.21.15.129:0/3431225903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb01a0ad0 con 0x7f1cb0105d20 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.630+0000 7f1cb4872640 1 -- 172.21.15.129:0/3431225903 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca400de50 con 0x7f1cb0105d20 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.630+0000 7f1cb4872640 1 -- 172.21.15.129:0/3431225903 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4004ce0 con 0x7f1cb0105d20 2024-04-26T21:19:12.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.630+0000 7f1cb5874640 1 -- 172.21.15.129:0/3431225903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb01a0fd0 con 0x7f1cb0105d20 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.633+0000 7f1cb4872640 1 -- 172.21.15.129:0/3431225903 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1ca4004e80 con 0x7f1cb0105d20 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.633+0000 7f1cb8583740 1 Processor -- start 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cb8583740 1 -- start start 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 0x55892ef19230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef1a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 0x55892ef1b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cb8583740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55892ee7dd80 con 0x55892ef1a030 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cb8583740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55892ee378b0 con 0x55892ef1a9d0 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cb8583740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55892ee79d40 con 0x55892ef18e10 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cb4872640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1c980a3c40 0x7f1c980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef1a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef1a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58050/0 (socket says 172.21.15.129:58050) 2024-04-26T21:19:12.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caeffd640 1 -- 172.21.15.129:0/1671564146 learned_addr learned my addr 172.21.15.129:0/1671564146 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/1671564146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 0x55892ef1b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caf7fe640 1 --2- 172.21.15.129:0/1671564146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 0x55892ef19230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1cb4872640 1 -- 172.21.15.129:0/3431225903 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f1ca40cffe0 con 0x7f1cb0105d20 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caf7fe640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1c980a3c40 0x7f1c980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caeffd640 1 -- 172.21.15.129:0/1671564146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 msgr2=0x55892ef19230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caeffd640 1 --2- 172.21.15.129:0/1671564146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 0x55892ef19230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caeffd640 1 -- 172.21.15.129:0/1671564146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 msgr2=0x55892ef1b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caeffd640 1 --2- 172.21.15.129:0/1671564146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 0x55892ef1b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.634+0000 7f1caeffd640 1 -- 172.21.15.129:0/1671564146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55892ee08a40 con 0x55892ef1a030 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.635+0000 7f1caeffd640 1 --2- 172.21.15.129:0/1671564146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef1a490 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000bd10 tx=0x7f1ca00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f4e14f9c17267b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.635+0000 7f1caf7fe640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1c980a3c40 0x7f1c980a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8005e30 tx=0x7f1ca80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.635+0000 7f1c95ffb640 1 -- 172.21.15.129:0/1671564146 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0004030 con 0x55892ef1a030 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.635+0000 7f1c95ffb640 1 -- 172.21.15.129:0/1671564146 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca00035f0 con 0x55892ef1a030 2024-04-26T21:19:12.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.635+0000 7f1c95ffb640 1 -- 172.21.15.129:0/1671564146 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00117b0 con 0x55892ef1a030 2024-04-26T21:19:12.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 -- 172.21.15.129:0/1671564146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 msgr2=0x55892ef1a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 --2- 172.21.15.129:0/1671564146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef1a490 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000bd10 tx=0x7f1ca00107a0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 -- 172.21.15.129:0/1671564146 shutdown_connections 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 --2- 172.21.15.129:0/1671564146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 0x55892ef1b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 --2- 172.21.15.129:0/1671564146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef1a490 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 --2- 172.21.15.129:0/1671564146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 0x55892ef19230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 -- 172.21.15.129:0/1671564146 >> 172.21.15.129:0/1671564146 conn(0x55892ef143d0 msgr2=0x55892ef147d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 -- 172.21.15.129:0/1671564146 shutdown_connections 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 -- 172.21.15.129:0/1671564146 wait complete. 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 -- start start 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.636+0000 7f1cb8583740 1 Processor -- start 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 0x55892ef2bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef2c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 0x55892ef330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cb8583740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55892ef1b9f0 con 0x55892ef1a030 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cb8583740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55892ee04d10 con 0x55892ef1a9d0 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cb8583740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55892ef1bb70 con 0x55892ef18e10 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cae7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef2c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cae7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef2c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58066/0 (socket says 172.21.15.129:58066) 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cae7fc640 1 -- 172.21.15.129:0/2904235927 learned_addr learned my addr 172.21.15.129:0/2904235927 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cb8583740 10 client.0 _ll_register_callbacks cb 0x55892ee31210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1caeffd640 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 0x55892ef2bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1caf7fe640 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 0x55892ef330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cae7fc640 1 -- 172.21.15.129:0/2904235927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 msgr2=0x55892ef2bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 0x55892ef2bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cae7fc640 1 -- 172.21.15.129:0/2904235927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 msgr2=0x55892ef330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 0x55892ef330e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cae7fc640 1 -- 172.21.15.129:0/2904235927 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55892ec597d0 con 0x55892ef1a030 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef2c0a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ca409c080 tx=0x7f1ca409bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f06dd36db66072 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.637+0000 7f1c8effd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1c8effd640 1 -- 172.21.15.129:0/2904235927 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4020040 con 0x55892ef1a030 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1c8effd640 1 -- 172.21.15.129:0/2904235927 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f1ca4098760 con 0x55892ef1a030 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1cb8583740 1 -- 172.21.15.129:0/2904235927 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca00035f0 con 0x55892ef1a030 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1c8effd640 1 -- 172.21.15.129:0/2904235927 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1ca40a4d60 con 0x55892ef1a030 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1cb8583740 10 client.15468 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1cb8583740 1 -- 172.21.15.129:0/2904235927 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55892eb8f3f0 con 0x55892ef1a030 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1c8effd640 1 -- 172.21.15.129:0/2904235927 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca00035f0 con 0x55892ef1a030 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1c8effd640 1 -- 172.21.15.129:0/2904235927 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f1ca4097e40 con 0x55892ef1a030 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1cb8583740 10 client.15468 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1cb8583740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:12.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1cb8583740 20 client.15468 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55892ef378e0 0x55892ef39d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1cb8583740 4 client.15468 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.638+0000 7f1cb8583740 1 -- 172.21.15.129:0/2904235927 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55892ee8b990 con 0x55892ef378e0 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.639+0000 7f1caeffd640 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55892ef378e0 0x55892ef39d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.639+0000 7f1caeffd640 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55892ef378e0 0x55892ef39d60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e78ffcf19ed3cd server_cookie=a9ddecec2a65fba6 in_seq=0 out_seq=0 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.639+0000 7f1c8effd640 10 client.15468 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1c8effd640 1 -- 172.21.15.129:0/2904235927 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55892ee8b990 con 0x55892ef378e0 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1c8effd640 10 client.15468 handle_command_reply: tid=0 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 client.15468 shutdown 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 -- 172.21.15.129:0/2904235927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 msgr2=0x55892ef2c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef2c0a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ca409c080 tx=0x7f1ca409bfe0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 -- 172.21.15.129:0/2904235927 shutdown_connections 2024-04-26T21:19:12.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55892ef378e0 0x55892ef39d60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a9d0 0x55892ef330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef1a030 0x55892ef2c0a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2904235927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef18e10 0x55892ef2bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 -- 172.21.15.129:0/2904235927 >> 172.21.15.129:0/2904235927 conn(0x55892ef143d0 msgr2=0x55892ef17420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1c8effd640 0 client.15468 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:12.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 -- 172.21.15.129:0/2904235927 shutdown_connections 2024-04-26T21:19:12.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 1 -- 172.21.15.129:0/2904235927 wait complete. 2024-04-26T21:19:12.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.640+0000 7f1cb8583740 20 client.15468 trim_cache size 0 max 16384 2024-04-26T21:19:12.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.657+0000 7f1cb8583740 1 Processor -- start 2024-04-26T21:19:12.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1cb8583740 1 -- start start 2024-04-26T21:19:12.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef26c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 0x55892ef2bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 0x55892ef2c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1cb8583740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01a5ba0 con 0x55892ef18e10 2024-04-26T21:19:12.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1cb8583740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01a8420 con 0x55892ef1a030 2024-04-26T21:19:12.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1cb8583740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01a6210 con 0x55892ef1a9d0 2024-04-26T21:19:12.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef26c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef26c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58070/0 (socket says 172.21.15.129:58070) 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1caeffd640 1 -- 172.21.15.129:0/2196142727 learned_addr learned my addr 172.21.15.129:0/2196142727 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/2196142727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 0x55892ef2bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1caf7fe640 1 --2- 172.21.15.129:0/2196142727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 0x55892ef2c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1caeffd640 1 -- 172.21.15.129:0/2196142727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 msgr2=0x55892ef2c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1caeffd640 1 --2- 172.21.15.129:0/2196142727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 0x55892ef2c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1caeffd640 1 -- 172.21.15.129:0/2196142727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 msgr2=0x55892ef2bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1caeffd640 1 --2- 172.21.15.129:0/2196142727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 0x55892ef2bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.658+0000 7f1caeffd640 1 -- 172.21.15.129:0/2196142727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55892eb4f0f0 con 0x55892ef18e10 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.659+0000 7f1caeffd640 1 --2- 172.21.15.129:0/2196142727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef26c70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000baa0 tx=0x7f1ca0005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f07e0b10057308 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.659+0000 7f1c957fa640 1 -- 172.21.15.129:0/2196142727 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca002f070 con 0x55892ef18e10 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.659+0000 7f1c957fa640 1 -- 172.21.15.129:0/2196142727 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca00054f0 con 0x55892ef18e10 2024-04-26T21:19:12.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.659+0000 7f1c957fa640 1 -- 172.21.15.129:0/2196142727 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca002a360 con 0x55892ef18e10 2024-04-26T21:19:12.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 -- 172.21.15.129:0/2196142727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 msgr2=0x55892ef26c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2196142727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef26c70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000baa0 tx=0x7f1ca0005890 comp rx=0 tx=0).stop 2024-04-26T21:19:12.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 -- 172.21.15.129:0/2196142727 shutdown_connections 2024-04-26T21:19:12.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2196142727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 0x55892ef2bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2196142727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 0x55892ef2c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2196142727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef26c70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 -- 172.21.15.129:0/2196142727 >> 172.21.15.129:0/2196142727 conn(0x55892ef143d0 msgr2=0x55892ef398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 -- 172.21.15.129:0/2196142727 shutdown_connections 2024-04-26T21:19:12.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 -- 172.21.15.129:0/2196142727 wait complete. 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 -- start start 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.660+0000 7f1cb8583740 1 Processor -- start 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef3c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 0x55892ef3cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1cb8583740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 0x55892ef9bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1cb8583740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01a68e0 con 0x55892ef18e10 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1cb8583740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01a7d50 con 0x55892ef1a030 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1cb8583740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01a6fb0 con 0x55892ef1a9d0 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1cb8583740 10 client.0 _ll_register_callbacks cb 0x7f1ca4098420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef3c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef3c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58084/0 (socket says 172.21.15.129:58084) 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caeffd640 1 -- 172.21.15.129:0/2789463928 learned_addr learned my addr 172.21.15.129:0/2789463928 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 0x55892ef3cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caf7fe640 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 0x55892ef9bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caeffd640 1 -- 172.21.15.129:0/2789463928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 msgr2=0x55892ef9bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caeffd640 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 0x55892ef9bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caeffd640 1 -- 172.21.15.129:0/2789463928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 msgr2=0x55892ef3cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caeffd640 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 0x55892ef3cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caeffd640 1 -- 172.21.15.129:0/2789463928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55892ee3e4b0 con 0x55892ef18e10 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 0x55892ef3cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1caeffd640 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef3c6c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1ca002d040 tx=0x7f1ca000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8338f8b695bf9a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.661+0000 7f1c8effd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.662+0000 7f1c8effd640 1 -- 172.21.15.129:0/2789463928 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0038060 con 0x55892ef18e10 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.662+0000 7f1c8effd640 1 -- 172.21.15.129:0/2789463928 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f1ca00052b0 con 0x55892ef18e10 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.662+0000 7f1cb8583740 1 -- 172.21.15.129:0/2789463928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca00054f0 con 0x55892ef18e10 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.662+0000 7f1c8effd640 1 -- 172.21.15.129:0/2789463928 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1ca002f070 con 0x55892ef18e10 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.662+0000 7f1cb8583740 10 client.15474 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:12.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.662+0000 7f1cb8583740 1 -- 172.21.15.129:0/2789463928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55892ef9d080 con 0x55892ef18e10 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.662+0000 7f1c8effd640 1 -- 172.21.15.129:0/2789463928 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca00054f0 con 0x55892ef18e10 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.663+0000 7f1c8effd640 1 -- 172.21.15.129:0/2789463928 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f1ca0033450 con 0x55892ef18e10 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.663+0000 7f1cb8583740 10 client.15474 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.663+0000 7f1cb8583740 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.663+0000 7f1cb8583740 20 client.15474 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.663+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55892ef9d7b0 0x55892ef9fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.663+0000 7f1cb8583740 4 client.15474 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.663+0000 7f1cb8583740 1 -- 172.21.15.129:0/2789463928 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55892ee85800 con 0x55892ef9d7b0 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.663+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55892ef9d7b0 0x55892ef9fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1c8effd640 10 client.15474 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cae7fc640 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55892ef9d7b0 0x55892ef9fbd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e832f689b2e7608f server_cookie=783949cc9154be2d in_seq=0 out_seq=0 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1c8effd640 1 -- 172.21.15.129:0/2789463928 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55892ee85800 con 0x55892ef9d7b0 2024-04-26T21:19:12.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1c8effd640 10 client.15474 handle_command_reply: tid=0 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 client.15474 shutdown 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 -- 172.21.15.129:0/2789463928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 msgr2=0x55892ef3c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef3c6c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1ca002d040 tx=0x7f1ca000b700 comp rx=0 tx=0).stop 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 -- 172.21.15.129:0/2789463928 shutdown_connections 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55892ef9d7b0 0x55892ef9fbd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55892ef1a030 0x55892ef3cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55892ef1a9d0 0x55892ef9bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 --2- 172.21.15.129:0/2789463928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55892ef18e10 0x55892ef3c6c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 -- 172.21.15.129:0/2789463928 >> 172.21.15.129:0/2789463928 conn(0x55892ef143d0 msgr2=0x55892ef26c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1c8effd640 0 client.15474 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 -- 172.21.15.129:0/2789463928 shutdown_connections 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.664+0000 7f1cb8583740 1 -- 172.21.15.129:0/2789463928 wait complete. 2024-04-26T21:19:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb8583740 20 client.15474 trim_cache size 0 max 16384 2024-04-26T21:19:12.671 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:19:12.671 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:19:12.671 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:19:12.671 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 -- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1c980a3c40 msgr2=0x7f1c980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1c980a3c40 0x7f1c980a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8005e30 tx=0x7f1ca80095c0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 -- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0105d20 msgr2=0x7f1cb0199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0105d20 0x7f1cb0199100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400d740 tx=0x7f1ca400dc00 comp rx=0 tx=0).stop 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 -- 172.21.15.129:0/3431225903 shutdown_connections 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f1c980a3c40 0x7f1c980a6100 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb0108650 0x7f1cb01a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cb0105d20 0x7f1cb0199100 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 --2- 172.21.15.129:0/3431225903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb01033f0 0x7f1cb0198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 -- 172.21.15.129:0/3431225903 >> 172.21.15.129:0/3431225903 conn(0x7f1cb00fcde0 msgr2=0x7f1cb0103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.665+0000 7f1cb5874640 1 -- 172.21.15.129:0/3431225903 shutdown_connections 2024-04-26T21:19:12.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.666+0000 7f1cb5874640 1 -- 172.21.15.129:0/3431225903 wait complete. 2024-04-26T21:19:12.677 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:12 smithi129 ceph-mon[29237]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:12.679 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:19:12.679 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:12.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:12 smithi171 ceph-mon[33478]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:12.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.780+0000 7f9a88e83640 1 -- 172.21.15.129:0/3995053077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84103420 msgr2=0x7f9a84109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.780+0000 7f9a88e83640 1 --2- 172.21.15.129:0/3995053077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84103420 0x7f9a84109cb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009ca0 tx=0x7f9a7c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.780+0000 7f9a88e83640 1 -- 172.21.15.129:0/3995053077 shutdown_connections 2024-04-26T21:19:12.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.780+0000 7f9a88e83640 1 --2- 172.21.15.129:0/3995053077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a84102a60 0x7f9a84102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.780+0000 7f9a88e83640 1 --2- 172.21.15.129:0/3995053077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84103420 0x7f9a84109cb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.780+0000 7f9a88e83640 1 --2- 172.21.15.129:0/3995053077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a84101870 0x7f9a84101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.780+0000 7f9a88e83640 1 -- 172.21.15.129:0/3995053077 >> 172.21.15.129:0/3995053077 conn(0x7f9a840fcde0 msgr2=0x7f9a840ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.780+0000 7f9a88e83640 1 -- 172.21.15.129:0/3995053077 shutdown_connections 2024-04-26T21:19:12.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.781+0000 7f9a88e83640 1 -- 172.21.15.129:0/3995053077 wait complete. 2024-04-26T21:19:12.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.781+0000 7f9a88e83640 1 Processor -- start 2024-04-26T21:19:12.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.781+0000 7f9a88e83640 1 -- start start 2024-04-26T21:19:12.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a88e83640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a84101870 0x7f9a84197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a88e83640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a84102a60 0x7f9a84197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a88e83640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84103420 0x7f9a8419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a88e83640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8410d3a0 con 0x7f9a84103420 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a88e83640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8410d220 con 0x7f9a84102a60 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a88e83640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8410d520 con 0x7f9a84101870 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a82d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84103420 0x7f9a8419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a81d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a84102a60 0x7f9a84197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a81d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a84102a60 0x7f9a84197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:53890/0 (socket says 172.21.15.129:53890) 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a81d9b640 1 -- 172.21.15.129:0/618124295 learned_addr learned my addr 172.21.15.129:0/618124295 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a8259c640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a84101870 0x7f9a84197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a81d9b640 1 -- 172.21.15.129:0/618124295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a84101870 msgr2=0x7f9a84197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a81d9b640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a84101870 0x7f9a84197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a81d9b640 1 -- 172.21.15.129:0/618124295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84103420 msgr2=0x7f9a8419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a81d9b640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84103420 0x7f9a8419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.782+0000 7f9a81d9b640 1 -- 172.21.15.129:0/618124295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8419f660 con 0x7f9a84102a60 2024-04-26T21:19:12.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.783+0000 7f9a81d9b640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a84102a60 0x7f9a84197f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800e9f0 tx=0x7f9a7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.783+0000 7f9a737fe640 1 -- 172.21.15.129:0/618124295 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7800cd80 con 0x7f9a84102a60 2024-04-26T21:19:12.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.783+0000 7f9a737fe640 1 -- 172.21.15.129:0/618124295 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a780108e0 con 0x7f9a84102a60 2024-04-26T21:19:12.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.783+0000 7f9a737fe640 1 -- 172.21.15.129:0/618124295 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78018980 con 0x7f9a84102a60 2024-04-26T21:19:12.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.783+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8419f950 con 0x7f9a84102a60 2024-04-26T21:19:12.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.783+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8419fe50 con 0x7f9a84102a60 2024-04-26T21:19:12.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.784+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a84108c10 con 0x7f9a84102a60 2024-04-26T21:19:12.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.786+0000 7f9a737fe640 1 -- 172.21.15.129:0/618124295 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9a78010430 con 0x7f9a84102a60 2024-04-26T21:19:12.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.787+0000 7f9a737fe640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9a6c0a3c40 0x7f9a6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.787+0000 7f9a8259c640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9a6c0a3c40 0x7f9a6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.787+0000 7f9a737fe640 1 -- 172.21.15.129:0/618124295 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9a780d0b60 con 0x7f9a84102a60 2024-04-26T21:19:12.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.787+0000 7f9a8259c640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9a6c0a3c40 0x7f9a6c0a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9a841028c0 tx=0x7f9a74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:12.790+0000 7f9a737fe640 1 -- 172.21.15.129:0/618124295 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9a7809cd20 con 0x7f9a84102a60 2024-04-26T21:19:12.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:12 smithi150 ceph-mon[32290]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:13.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.013+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a8402dc90 con 0x7f9a84102a60 2024-04-26T21:19:13.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.014+0000 7f9a737fe640 1 -- 172.21.15.129:0/618124295 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f9a780a1020 con 0x7f9a84102a60 2024-04-26T21:19:13.015 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:19:13.015 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:19:13.016 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9a6c0a3c40 msgr2=0x7f9a6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9a6c0a3c40 0x7f9a6c0a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9a841028c0 tx=0x7f9a74005f90 comp rx=0 tx=0).stop 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a84102a60 msgr2=0x7f9a84197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a84102a60 0x7f9a84197f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800e9f0 tx=0x7f9a7800eeb0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 shutdown_connections 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9a6c0a3c40 0x7f9a6c0a6100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a84102a60 0x7f9a84197f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84103420 0x7f9a8419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 --2- 172.21.15.129:0/618124295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a84101870 0x7f9a84197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 >> 172.21.15.129:0/618124295 conn(0x7f9a840fcde0 msgr2=0x7f9a84105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.016+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 shutdown_connections 2024-04-26T21:19:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.017+0000 7f9a88e83640 1 -- 172.21.15.129:0/618124295 wait complete. 2024-04-26T21:19:13.029 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:19:13.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.129+0000 7f50c299d640 1 -- 172.21.15.129:0/964509297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc101870 msgr2=0x7f50bc101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.129+0000 7f50c299d640 1 --2- 172.21.15.129:0/964509297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc101870 0x7f50bc101c90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009900 tx=0x7f50ac02a410 comp rx=0 tx=0).stop 2024-04-26T21:19:13.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.129+0000 7f50c299d640 1 -- 172.21.15.129:0/964509297 shutdown_connections 2024-04-26T21:19:13.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.129+0000 7f50c299d640 1 --2- 172.21.15.129:0/964509297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc102a60 0x7f50bc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.129+0000 7f50c299d640 1 --2- 172.21.15.129:0/964509297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f50bc103420 0x7f50bc109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.129+0000 7f50c299d640 1 --2- 172.21.15.129:0/964509297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc101870 0x7f50bc101c90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.129+0000 7f50c299d640 1 -- 172.21.15.129:0/964509297 >> 172.21.15.129:0/964509297 conn(0x7f50bc0fcde0 msgr2=0x7f50bc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.130+0000 7f50c299d640 1 -- 172.21.15.129:0/964509297 shutdown_connections 2024-04-26T21:19:13.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.130+0000 7f50c299d640 1 -- 172.21.15.129:0/964509297 wait complete. 2024-04-26T21:19:13.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.130+0000 7f50c299d640 1 Processor -- start 2024-04-26T21:19:13.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.130+0000 7f50c299d640 1 -- start start 2024-04-26T21:19:13.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50c299d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f50bc101870 0x7f50bc1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50c299d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc102a60 0x7f50bc196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50c299d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc103420 0x7f50bc19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc102a60 0x7f50bc196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50c299d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc10c2a0 con 0x7f50bc102a60 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc102a60 0x7f50bc196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58138/0 (socket says 172.21.15.129:58138) 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bb7fe640 1 -- 172.21.15.129:0/1162466165 learned_addr learned my addr 172.21.15.129:0/1162466165 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc10c120 con 0x7f50bc103420 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc10c420 con 0x7f50bc101870 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bbfff640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f50bc101870 0x7f50bc1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc103420 0x7f50bc19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bb7fe640 1 -- 172.21.15.129:0/1162466165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f50bc101870 msgr2=0x7f50bc1969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f50bc101870 0x7f50bc1969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bb7fe640 1 -- 172.21.15.129:0/1162466165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc103420 msgr2=0x7f50bc19df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc103420 0x7f50bc19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bb7fe640 1 -- 172.21.15.129:0/1162466165 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc1a29f0 con 0x7f50bc102a60 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.131+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc102a60 0x7f50bc196f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f50b000e9f0 tx=0x7f50b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.132+0000 7f50b97fa640 1 -- 172.21.15.129:0/1162466165 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b000cd80 con 0x7f50bc102a60 2024-04-26T21:19:13.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.132+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc1a2ce0 con 0x7f50bc102a60 2024-04-26T21:19:13.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.132+0000 7f50b97fa640 1 -- 172.21.15.129:0/1162466165 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b00108e0 con 0x7f50bc102a60 2024-04-26T21:19:13.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.132+0000 7f50b97fa640 1 -- 172.21.15.129:0/1162466165 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0018980 con 0x7f50bc102a60 2024-04-26T21:19:13.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.132+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc1a3220 con 0x7f50bc102a60 2024-04-26T21:19:13.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.134+0000 7f50c578f740 1 Processor -- start 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.134+0000 7f50c578f740 1 -- start start 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.134+0000 7f50c578f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418c820 0x558c9418cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.134+0000 7f50c578f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c9418de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.134+0000 7f50c578f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 0x558c94194c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.134+0000 7f50c578f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558c940f0d80 con 0x558c9418e3b0 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.134+0000 7f50c578f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558c940aaa20 con 0x558c9418d9f0 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.134+0000 7f50c578f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558c940ecd40 con 0x558c9418c820 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50bbfff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c9418de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50bbfff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c9418de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:53912/0 (socket says 172.21.15.129:53912) 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50bbfff640 1 -- 172.21.15.129:0/4095962857 learned_addr learned my addr 172.21.15.129:0/4095962857 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/4095962857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418c820 0x558c9418cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50b97fa640 1 -- 172.21.15.129:0/1162466165 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f50b0010430 con 0x7f50bc102a60 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50bbfff640 1 -- 172.21.15.129:0/4095962857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418c820 msgr2=0x558c9418cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50bbfff640 1 --2- 172.21.15.129:0/4095962857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418c820 0x558c9418cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50bbfff640 1 -- 172.21.15.129:0/4095962857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 msgr2=0x558c94194c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50bbfff640 1 --2- 172.21.15.129:0/4095962857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 0x558c94194c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50bbfff640 1 -- 172.21.15.129:0/4095962857 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c940b5c30 con 0x558c9418d9f0 2024-04-26T21:19:13.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.135+0000 7f50bbfff640 1 --2- 172.21.15.129:0/4095962857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c9418de70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009900 tx=0x7f50ac032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6643bb043848cc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.136+0000 7f50a27fc640 1 -- 172.21.15.129:0/4095962857 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac03f070 con 0x558c9418d9f0 2024-04-26T21:19:13.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.136+0000 7f50a27fc640 1 -- 172.21.15.129:0/4095962857 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ac004040 con 0x558c9418d9f0 2024-04-26T21:19:13.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.136+0000 7f50a27fc640 1 -- 172.21.15.129:0/4095962857 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac035070 con 0x558c9418d9f0 2024-04-26T21:19:13.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.136+0000 7f50b97fa640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f50a40a3c40 0x7f50a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.136+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f50a40a3c40 0x7f50a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.136+0000 7f50b97fa640 1 -- 172.21.15.129:0/1162466165 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f50b00d09d0 con 0x7f50bc102a60 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 -- 172.21.15.129:0/4095962857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 msgr2=0x558c9418de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f50a40a3c40 0x7f50a40a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x558c9418d8f0 tx=0x7f50b400f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 --2- 172.21.15.129:0/4095962857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c9418de70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009900 tx=0x7f50ac032b20 comp rx=0 tx=0).stop 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 -- 172.21.15.129:0/4095962857 shutdown_connections 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 --2- 172.21.15.129:0/4095962857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c9418de70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 --2- 172.21.15.129:0/4095962857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 0x558c94194c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 --2- 172.21.15.129:0/4095962857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418c820 0x558c9418cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 -- 172.21.15.129:0/4095962857 >> 172.21.15.129:0/4095962857 conn(0x558c940cd910 msgr2=0x558c94188120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 -- 172.21.15.129:0/4095962857 shutdown_connections 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 -- 172.21.15.129:0/4095962857 wait complete. 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 -- start start 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.137+0000 7f50c578f740 1 Processor -- start 2024-04-26T21:19:13.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50c578f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418c820 0x558c94196b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50c578f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c94197060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418c820 0x558c94196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50c578f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 0x558c9419e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418c820 0x558c94196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58158/0 (socket says 172.21.15.129:58158) 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bbfff640 1 -- 172.21.15.129:0/467144766 learned_addr learned my addr 172.21.15.129:0/467144766 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558c94195360 con 0x558c9418c820 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558c94077dc0 con 0x558c9418d9f0 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558c941954e0 con 0x558c9418e3b0 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c94197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50c578f740 10 client.0 _ll_register_callbacks cb 0x558c94093600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 0x558c9419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bb7fe640 1 -- 172.21.15.129:0/467144766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 msgr2=0x558c9419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 0x558c9419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bb7fe640 1 -- 172.21.15.129:0/467144766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418c820 msgr2=0x558c94196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418c820 0x558c94196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bb7fe640 1 -- 172.21.15.129:0/467144766 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c93e023f0 con 0x558c9418d9f0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50bbfff640 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418c820 0x558c94196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.138+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 0x558c9419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c94197060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50b00ce370 tx=0x7f50b0010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac9520003297a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f509b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f509b7fe640 1 -- 172.21.15.129:0/467144766 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00a57b0 con 0x558c9418d9f0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f509b7fe640 1 -- 172.21.15.129:0/467144766 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f50b0010a90 con 0x558c9418d9f0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50ac004040 con 0x558c9418d9f0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f509b7fe640 1 -- 172.21.15.129:0/467144766 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f50b009bf90 con 0x558c9418d9f0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50c578f740 10 client.25097 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558c93ecc7d0 con 0x558c9418d9f0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f509b7fe640 1 -- 172.21.15.129:0/467144766 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ac004040 con 0x558c9418d9f0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f509b7fe640 1 -- 172.21.15.129:0/467144766 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f50b0098a90 con 0x558c9418d9f0 2024-04-26T21:19:13.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50c578f740 10 client.25097 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50c578f740 10 client.25097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50c578f740 20 client.25097 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50c578f740 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x558c941a27a0 0x558c941a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50c578f740 4 client.25097 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.139+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c940fe990 con 0x558c941a27a0 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.140+0000 7f50bbfff640 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x558c941a27a0 0x558c941a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.140+0000 7f509b7fe640 10 client.25097 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.140+0000 7f50bbfff640 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x558c941a27a0 0x558c941a4c00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b8f0f358c3483a server_cookie=a3f7200e5ad9236c in_seq=0 out_seq=0 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f509b7fe640 1 -- 172.21.15.129:0/467144766 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c940fe990 con 0x558c941a27a0 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f509b7fe640 10 client.25097 handle_command_reply: tid=0 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f50c578f740 1 client.25097 shutdown 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 msgr2=0x558c94197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f50c578f740 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c94197060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50b00ce370 tx=0x7f50b0010860 comp rx=0 tx=0).stop 2024-04-26T21:19:13.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 shutdown_connections 2024-04-26T21:19:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f50c578f740 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x558c941a27a0 0x558c941a4c00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f509b7fe640 0 client.25097 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f50c578f740 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418d9f0 0x558c94197060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f50c578f740 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 0x558c9419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f50c578f740 1 --2- 172.21.15.129:0/467144766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418c820 0x558c94196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.141+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 >> 172.21.15.129:0/467144766 conn(0x558c940cd910 msgr2=0x558c94187280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.142+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 shutdown_connections 2024-04-26T21:19:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.142+0000 7f50c578f740 1 -- 172.21.15.129:0/467144766 wait complete. 2024-04-26T21:19:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.142+0000 7f50c578f740 20 client.25097 trim_cache size 0 max 16384 2024-04-26T21:19:13.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c578f740 1 Processor -- start 2024-04-26T21:19:13.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c578f740 1 -- start start 2024-04-26T21:19:13.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c578f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 0x558c941830e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c578f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418d9f0 0x558c9419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c578f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 0x558c9418afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c578f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1aa000 con 0x558c9418e3b0 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c578f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1ac880 con 0x558c9418c820 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c578f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1aa670 con 0x558c9418d9f0 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c0f3a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 0x558c9418afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c0f3a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 0x558c9418afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58166/0 (socket says 172.21.15.129:58166) 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.167+0000 7f50c0f3a640 1 -- 172.21.15.129:0/1389694207 learned_addr learned my addr 172.21.15.129:0/1389694207 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50bbfff640 1 --2- 172.21.15.129:0/1389694207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 0x558c941830e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/1389694207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418d9f0 0x558c9419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50c0f3a640 1 -- 172.21.15.129:0/1389694207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418d9f0 msgr2=0x558c9419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/1389694207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418d9f0 0x558c9419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50c0f3a640 1 -- 172.21.15.129:0/1389694207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 msgr2=0x558c941830e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/1389694207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 0x558c941830e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50c0f3a640 1 -- 172.21.15.129:0/1389694207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c941838b0 con 0x558c9418e3b0 2024-04-26T21:19:13.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/1389694207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 0x558c9418afe0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f50b4019610 tx=0x7f50b4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b16b8f555386b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50a1ffb640 1 -- 172.21.15.129:0/1389694207 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4017500 con 0x558c9418e3b0 2024-04-26T21:19:13.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50a1ffb640 1 -- 172.21.15.129:0/1389694207 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b402a070 con 0x558c9418e3b0 2024-04-26T21:19:13.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.168+0000 7f50a1ffb640 1 -- 172.21.15.129:0/1389694207 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4025a70 con 0x558c9418e3b0 2024-04-26T21:19:13.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 -- 172.21.15.129:0/1389694207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 msgr2=0x558c9418afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 --2- 172.21.15.129:0/1389694207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 0x558c9418afe0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f50b4019610 tx=0x7f50b4018000 comp rx=0 tx=0).stop 2024-04-26T21:19:13.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 -- 172.21.15.129:0/1389694207 shutdown_connections 2024-04-26T21:19:13.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 --2- 172.21.15.129:0/1389694207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418e3b0 0x558c9418afe0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 --2- 172.21.15.129:0/1389694207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418d9f0 0x558c9419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 --2- 172.21.15.129:0/1389694207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 0x558c941830e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 -- 172.21.15.129:0/1389694207 >> 172.21.15.129:0/1389694207 conn(0x558c940cd910 msgr2=0x558c94187ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 -- 172.21.15.129:0/1389694207 shutdown_connections 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 -- 172.21.15.129:0/1389694207 wait complete. 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.169+0000 7f50c578f740 1 -- start start 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50c578f740 1 Processor -- start 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50c578f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 0x558c9419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50c578f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418d9f0 0x558c9419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50c578f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 0x558c9419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50c578f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1aad40 con 0x558c9418d9f0 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50c578f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1ac1b0 con 0x558c9418c820 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50c578f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1ab410 con 0x558c9418e3b0 2024-04-26T21:19:13.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50c578f740 10 client.0 _ll_register_callbacks cb 0x7f50b0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50bb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418d9f0 0x558c9419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50bb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418d9f0 0x558c9419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58174/0 (socket says 172.21.15.129:58174) 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50bb7fe640 1 -- 172.21.15.129:0/3792435162 learned_addr learned my addr 172.21.15.129:0/3792435162 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50bbfff640 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 0x558c9419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50c0f3a640 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 0x558c9419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50bb7fe640 1 -- 172.21.15.129:0/3792435162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 msgr2=0x558c9419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 0x558c9419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50bb7fe640 1 -- 172.21.15.129:0/3792435162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 msgr2=0x558c9419b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 0x558c9419b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.170+0000 7f50bb7fe640 1 -- 172.21.15.129:0/3792435162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c9419d1e0 con 0x558c9418d9f0 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50bb7fe640 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418d9f0 0x558c9419bbe0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50b0098fa0 tx=0x7f50b001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cd86566d2868c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f509b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f509b7fe640 1 -- 172.21.15.129:0/3792435162 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00a57b0 con 0x558c9418d9f0 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f509b7fe640 1 -- 172.21.15.129:0/3792435162 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f50b0018d20 con 0x558c9418d9f0 2024-04-26T21:19:13.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50c578f740 1 -- 172.21.15.129:0/3792435162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b402a070 con 0x558c9418d9f0 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f509b7fe640 1 -- 172.21.15.129:0/3792435162 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f50b009c4a0 con 0x558c9418d9f0 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50c578f740 10 client.15495 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50c578f740 1 -- 172.21.15.129:0/3792435162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558c94208db0 con 0x558c9418d9f0 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f509b7fe640 1 -- 172.21.15.129:0/3792435162 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b00a2080 con 0x558c9418d9f0 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f509b7fe640 1 -- 172.21.15.129:0/3792435162 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f50b0087170 con 0x558c9418d9f0 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50c578f740 10 client.15495 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50c578f740 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50c578f740 20 client.15495 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50c578f740 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x558c941a27a0 0x558c9420b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50c578f740 4 client.15495 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.171+0000 7f50c578f740 1 -- 172.21.15.129:0/3792435162 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f50b402a070 con 0x558c941a27a0 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.172+0000 7f50bbfff640 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x558c941a27a0 0x558c9420b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.172+0000 7f509b7fe640 10 client.15495 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.172+0000 7f50bbfff640 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x558c941a27a0 0x558c9420b610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc0c2a471542bed server_cookie=aec8e804d4d060ec in_seq=0 out_seq=0 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.172+0000 7f509b7fe640 1 -- 172.21.15.129:0/3792435162 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f50b402a070 con 0x558c941a27a0 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.172+0000 7f509b7fe640 10 client.15495 handle_command_reply: tid=0 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.172+0000 7f50c578f740 1 client.15495 shutdown 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.172+0000 7f50c578f740 1 -- 172.21.15.129:0/3792435162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418d9f0 msgr2=0x558c9419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.172+0000 7f50c578f740 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418d9f0 0x558c9419bbe0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50b0098fa0 tx=0x7f50b001f000 comp rx=0 tx=0).stop 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c578f740 1 -- 172.21.15.129:0/3792435162 shutdown_connections 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c578f740 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x558c941a27a0 0x558c9420b610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f509b7fe640 0 client.15495 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c578f740 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558c9418e3b0 0x558c9419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c578f740 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c9418d9f0 0x558c9419bbe0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c578f740 1 --2- 172.21.15.129:0/3792435162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558c9418c820 0x558c9419b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c578f740 1 -- 172.21.15.129:0/3792435162 >> 172.21.15.129:0/3792435162 conn(0x558c940cd910 msgr2=0x558c941869f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c578f740 1 -- 172.21.15.129:0/3792435162 shutdown_connections 2024-04-26T21:19:13.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c578f740 1 -- 172.21.15.129:0/3792435162 wait complete. 2024-04-26T21:19:13.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c578f740 20 client.15495 trim_cache size 0 max 16384 2024-04-26T21:19:13.180 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:19:13.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f50a40a3c40 msgr2=0x7f50a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f50a40a3c40 0x7f50a40a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x558c9418d8f0 tx=0x7f50b400f040 comp rx=0 tx=0).stop 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc102a60 msgr2=0x7f50bc196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc102a60 0x7f50bc196f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f50b000e9f0 tx=0x7f50b000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 shutdown_connections 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f50a40a3c40 0x7f50a40a6100 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc103420 0x7f50bc19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50bc102a60 0x7f50bc196f00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 --2- 172.21.15.129:0/1162466165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f50bc101870 0x7f50bc1969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 >> 172.21.15.129:0/1162466165 conn(0x7f50bc0fcde0 msgr2=0x7f50bc105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 shutdown_connections 2024-04-26T21:19:13.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:13.173+0000 7f50c299d640 1 -- 172.21.15.129:0/1162466165 wait complete. 2024-04-26T21:19:13.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:13 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/618124295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:19:13.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:13 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/618124295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:19:13.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:13 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/618124295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:19:14.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:19:14.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:14.186 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:19:14.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 -- 172.21.15.129:0/1022211107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 msgr2=0x7f2654101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 --2- 172.21.15.129:0/1022211107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 0x7f2654101c90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2644009900 tx=0x7f264402a410 comp rx=0 tx=0).stop 2024-04-26T21:19:14.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 -- 172.21.15.129:0/1022211107 shutdown_connections 2024-04-26T21:19:14.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 --2- 172.21.15.129:0/1022211107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2654102a60 0x7f2654102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 --2- 172.21.15.129:0/1022211107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2654103420 0x7f2654109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 --2- 172.21.15.129:0/1022211107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 0x7f2654101c90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 -- 172.21.15.129:0/1022211107 >> 172.21.15.129:0/1022211107 conn(0x7f26540fcde0 msgr2=0x7f26540ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 -- 172.21.15.129:0/1022211107 shutdown_connections 2024-04-26T21:19:14.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 -- 172.21.15.129:0/1022211107 wait complete. 2024-04-26T21:19:14.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.344+0000 7f265a30a640 1 Processor -- start 2024-04-26T21:19:14.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f265a30a640 1 -- start start 2024-04-26T21:19:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f265a30a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 0x7f2654198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f265a30a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2654102a60 0x7f2654199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f265a30a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2654103420 0x7f26541a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f265a30a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f265410e520 con 0x7f2654101870 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f265a30a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f265410e3a0 con 0x7f2654103420 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f265a30a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f265410e6a0 con 0x7f2654102a60 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f2653fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 0x7f2654198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f2653fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 0x7f2654198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58190/0 (socket says 172.21.15.129:58190) 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f2653fff640 1 -- 172.21.15.129:0/644295276 learned_addr learned my addr 172.21.15.129:0/644295276 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f26588a7640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2654103420 0x7f26541a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f26537fe640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2654102a60 0x7f2654199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f2653fff640 1 -- 172.21.15.129:0/644295276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2654102a60 msgr2=0x7f2654199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f2653fff640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2654102a60 0x7f2654199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f2653fff640 1 -- 172.21.15.129:0/644295276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2654103420 msgr2=0x7f26541a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.345+0000 7f2653fff640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2654103420 0x7f26541a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.346+0000 7f2653fff640 1 -- 172.21.15.129:0/644295276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26541a0800 con 0x7f2654101870 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.346+0000 7f2653fff640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 0x7f2654198be0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2644031500 tx=0x7f2644003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.346+0000 7f26517fa640 1 -- 172.21.15.129:0/644295276 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264403f070 con 0x7f2654101870 2024-04-26T21:19:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.346+0000 7f26517fa640 1 -- 172.21.15.129:0/644295276 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2644031c50 con 0x7f2654101870 2024-04-26T21:19:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.346+0000 7f265a30a640 1 -- 172.21.15.129:0/644295276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26541a0a90 con 0x7f2654101870 2024-04-26T21:19:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.346+0000 7f265a30a640 1 -- 172.21.15.129:0/644295276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26541a0fd0 con 0x7f2654101870 2024-04-26T21:19:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.346+0000 7f26517fa640 1 -- 172.21.15.129:0/644295276 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644035030 con 0x7f2654101870 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.349+0000 7f265d01e740 1 Processor -- start 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.349+0000 7f26517fa640 1 -- 172.21.15.129:0/644295276 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2644042070 con 0x7f2654101870 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f265d01e740 1 -- start start 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f265d01e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f265d01e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 0x55a8260b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f265d01e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f265d01e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a826019b00 con 0x55a8260ba410 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f265d01e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a825fb5920 con 0x55a8260b48b0 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f265d01e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a826015ac0 con 0x55a8260b52a0 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26537fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26588a7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26588a7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:53968/0 (socket says 172.21.15.129:53968) 2024-04-26T21:19:14.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26537fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260bc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58198/0 (socket says 172.21.15.129:58198) 2024-04-26T21:19:14.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26588a7640 1 -- 172.21.15.129:0/2625589283 learned_addr learned my addr 172.21.15.129:0/2625589283 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f2653fff640 1 --2- 172.21.15.129:0/2625589283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 0x55a8260b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26588a7640 1 -- 172.21.15.129:0/2625589283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 msgr2=0x55a8260b9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26588a7640 1 --2- 172.21.15.129:0/2625589283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 0x55a8260b9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26517fa640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f263c0a3cc0 0x7f263c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26588a7640 1 -- 172.21.15.129:0/2625589283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 msgr2=0x55a8260bc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26588a7640 1 --2- 172.21.15.129:0/2625589283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260bc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26588a7640 1 -- 172.21.15.129:0/2625589283 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a825df37e0 con 0x55a8260b48b0 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.350+0000 7f26537fe640 1 --2- 172.21.15.129:0/2625589283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260bc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.351+0000 7f26517fa640 1 -- 172.21.15.129:0/644295276 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f26440f0c80 con 0x7f2654101870 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.351+0000 7f26588a7640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f263c0a3cc0 0x7f263c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.351+0000 7f26588a7640 1 --2- 172.21.15.129:0/2625589283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260b4cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f264c009fd0 tx=0x7f264c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14bad8479adb8da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.351+0000 7f263a7fc640 1 -- 172.21.15.129:0/2625589283 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c00f6e0 con 0x55a8260b48b0 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.351+0000 7f26588a7640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f263c0a3cc0 0x7f263c0a6180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f264c022000 tx=0x7f264c023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.351+0000 7f263a7fc640 1 -- 172.21.15.129:0/2625589283 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f264c00fd30 con 0x55a8260b48b0 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.351+0000 7f263a7fc640 1 -- 172.21.15.129:0/2625589283 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c01bbc0 con 0x55a8260b48b0 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.352+0000 7f265d01e740 1 -- 172.21.15.129:0/2625589283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 msgr2=0x55a8260b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.352+0000 7f265d01e740 1 --2- 172.21.15.129:0/2625589283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260b4cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f264c009fd0 tx=0x7f264c0118d0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.352+0000 7f265d01e740 1 -- 172.21.15.129:0/2625589283 shutdown_connections 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.352+0000 7f265d01e740 1 --2- 172.21.15.129:0/2625589283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260bc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.352+0000 7f265d01e740 1 --2- 172.21.15.129:0/2625589283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 0x55a8260b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.352+0000 7f265d01e740 1 --2- 172.21.15.129:0/2625589283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260b4cd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.352+0000 7f265d01e740 1 -- 172.21.15.129:0/2625589283 >> 172.21.15.129:0/2625589283 conn(0x55a8260aff20 msgr2=0x55a8260b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.352+0000 7f265d01e740 1 -- 172.21.15.129:0/2625589283 shutdown_connections 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.352+0000 7f265d01e740 1 -- 172.21.15.129:0/2625589283 wait complete. 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.353+0000 7f265d01e740 1 -- start start 2024-04-26T21:19:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.353+0000 7f265d01e740 1 Processor -- start 2024-04-26T21:19:14.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.353+0000 7f265d01e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260b48b0 0x55a8260bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.353+0000 7f265d01e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 0x55a8260bf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f2653fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260b48b0 0x55a8260bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f2653fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260b48b0 0x55a8260bedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58202/0 (socket says 172.21.15.129:58202) 2024-04-26T21:19:14.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f2653fff640 1 -- 172.21.15.129:0/4007688344 learned_addr learned my addr 172.21.15.129:0/4007688344 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f265d01e740 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260ba410 0x55a8260c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a8260bd3c0 con 0x55a8260b48b0 2024-04-26T21:19:14.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a825fb35d0 con 0x55a8260b52a0 2024-04-26T21:19:14.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a8260bd540 con 0x55a8260ba410 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f26537fe640 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 0x55a8260bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f2653fff640 1 -- 172.21.15.129:0/4007688344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260ba410 msgr2=0x55a8260c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f2653fff640 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260ba410 0x55a8260c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f2653fff640 1 -- 172.21.15.129:0/4007688344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 msgr2=0x55a8260bf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f2653fff640 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 0x55a8260bf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f2653fff640 1 -- 172.21.15.129:0/4007688344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a825e53f70 con 0x55a8260b48b0 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f2653fff640 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260b48b0 0x55a8260bedc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2644005ec0 tx=0x7f264402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b20d0622249d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f26337fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f265d01e740 10 client.0 _ll_register_callbacks cb 0x55a825fa1080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f264c00fd30 con 0x55a8260b48b0 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.354+0000 7f26337fe640 1 -- 172.21.15.129:0/4007688344 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644049070 con 0x55a8260b48b0 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f26337fe640 1 -- 172.21.15.129:0/4007688344 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f26440bd100 con 0x55a8260b48b0 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f26337fe640 1 -- 172.21.15.129:0/4007688344 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f26440bc800 con 0x55a8260b48b0 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f265d01e740 10 client.15507 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55a825ceb170 con 0x55a8260b48b0 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f26337fe640 1 -- 172.21.15.129:0/4007688344 <== mon.0 v2:172.21.15.129:3300/0 4 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f26440c55d0 con 0x55a8260b48b0 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f265d01e740 10 client.15507 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f265d01e740 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f265d01e740 20 client.15507 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f265d01e740 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55a8260ca640 0x55a8260ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f265d01e740 4 client.15507 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f264c00fd30 con 0x55a8260ca640 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.355+0000 7f26537fe640 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55a8260ca640 0x55a8260ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.356+0000 7f26337fe640 10 client.15507 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.356+0000 7f26537fe640 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55a8260ca640 0x55a8260ccad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9d3f7d92f7e72 server_cookie=e6b338e365ee9b0a in_seq=0 out_seq=0 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.357+0000 7f26337fe640 1 -- 172.21.15.129:0/4007688344 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f264c00fd30 con 0x55a8260ca640 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.357+0000 7f26337fe640 10 client.15507 handle_command_reply: tid=0 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.357+0000 7f265d01e740 1 client.15507 shutdown 2024-04-26T21:19:14.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.357+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260b48b0 msgr2=0x55a8260bedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.357+0000 7f265d01e740 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260b48b0 0x55a8260bedc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2644005ec0 tx=0x7f264402a410 comp rx=0 tx=0).stop 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 shutdown_connections 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f265d01e740 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55a8260ca640 0x55a8260ccad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f265d01e740 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 0x55a8260bf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f265d01e740 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260ba410 0x55a8260c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f26337fe640 0 client.15507 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f265d01e740 1 --2- 172.21.15.129:0/4007688344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260b48b0 0x55a8260bedc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 >> 172.21.15.129:0/4007688344 conn(0x55a8260aff20 msgr2=0x55a8260af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 shutdown_connections 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f265d01e740 1 -- 172.21.15.129:0/4007688344 wait complete. 2024-04-26T21:19:14.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.358+0000 7f265d01e740 20 client.15507 trim_cache size 0 max 16384 2024-04-26T21:19:14.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f265d01e740 1 Processor -- start 2024-04-26T21:19:14.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f265d01e740 1 -- start start 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f265d01e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b48b0 0x55a8260bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f265d01e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 0x55a8260c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f2653fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b48b0 0x55a8260bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f2653fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b48b0 0x55a8260bf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:60838/0 (socket says 172.21.15.129:60838) 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f26537fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 0x55a8260c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f26537fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 0x55a8260c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:53984/0 (socket says 172.21.15.129:53984) 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f265d01e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.392+0000 7f2653fff640 1 -- 172.21.15.129:0/2964722686 learned_addr learned my addr 172.21.15.129:0/2964722686 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f265d01e740 1 -- 172.21.15.129:0/2964722686 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f26541a5b90 con 0x55a8260ba410 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f265d01e740 1 -- 172.21.15.129:0/2964722686 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26541a8410 con 0x55a8260b52a0 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f265d01e740 1 -- 172.21.15.129:0/2964722686 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f26541a6200 con 0x55a8260b48b0 2024-04-26T21:19:14.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f26588a7640 1 --2- 172.21.15.129:0/2964722686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f2653fff640 1 -- 172.21.15.129:0/2964722686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 msgr2=0x55a8260c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f2653fff640 1 --2- 172.21.15.129:0/2964722686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 0x55a8260c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f2653fff640 1 -- 172.21.15.129:0/2964722686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 msgr2=0x55a8260b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f2653fff640 1 --2- 172.21.15.129:0/2964722686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f2653fff640 1 -- 172.21.15.129:0/2964722686 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8260afd50 con 0x55a8260b48b0 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f26588a7640 1 --2- 172.21.15.129:0/2964722686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f2653fff640 1 --2- 172.21.15.129:0/2964722686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b48b0 0x55a8260bf0e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26440f4570 tx=0x7f26440dd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6039618f2d840b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.393+0000 7f2639ffb640 1 -- 172.21.15.129:0/2964722686 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440c11a0 con 0x55a8260b48b0 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.394+0000 7f2639ffb640 1 -- 172.21.15.129:0/2964722686 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26440c5a60 con 0x55a8260b48b0 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.394+0000 7f2639ffb640 1 -- 172.21.15.129:0/2964722686 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264403b460 con 0x55a8260b48b0 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.394+0000 7f265d01e740 1 -- 172.21.15.129:0/2964722686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b48b0 msgr2=0x55a8260bf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.394+0000 7f265d01e740 1 --2- 172.21.15.129:0/2964722686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b48b0 0x55a8260bf0e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26440f4570 tx=0x7f26440dd040 comp rx=0 tx=0).stop 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.394+0000 7f265d01e740 1 -- 172.21.15.129:0/2964722686 shutdown_connections 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.394+0000 7f265d01e740 1 --2- 172.21.15.129:0/2964722686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b52a0 0x55a8260c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.395+0000 7f265d01e740 1 --2- 172.21.15.129:0/2964722686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.395+0000 7f265d01e740 1 --2- 172.21.15.129:0/2964722686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b48b0 0x55a8260bf0e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.395+0000 7f265d01e740 1 -- 172.21.15.129:0/2964722686 >> 172.21.15.129:0/2964722686 conn(0x55a8260aff20 msgr2=0x55a8260be9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.395+0000 7f265d01e740 1 -- 172.21.15.129:0/2964722686 shutdown_connections 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.395+0000 7f265d01e740 1 -- 172.21.15.129:0/2964722686 wait complete. 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.395+0000 7f265d01e740 1 -- start start 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.395+0000 7f265d01e740 1 Processor -- start 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.395+0000 7f265d01e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f265d01e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 0x55a8260c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f265d01e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f265d01e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f26541a68d0 con 0x55a8260ba410 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f265d01e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26541a7d40 con 0x55a8260b48b0 2024-04-26T21:19:14.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f2653fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f265d01e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f26541a6fa0 con 0x55a8260b52a0 2024-04-26T21:19:14.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f2653fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:53996/0 (socket says 172.21.15.129:53996) 2024-04-26T21:19:14.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f2653fff640 1 -- 172.21.15.129:0/2377561527 learned_addr learned my addr 172.21.15.129:0/2377561527 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f265d01e740 10 client.0 _ll_register_callbacks cb 0x7f26440b9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:14.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f26588a7640 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f26537fe640 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 0x55a8260c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f2653fff640 1 -- 172.21.15.129:0/2377561527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 msgr2=0x55a8260c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f2653fff640 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 0x55a8260c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f2653fff640 1 -- 172.21.15.129:0/2377561527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 msgr2=0x55a8260c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f2653fff640 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f2653fff640 1 -- 172.21.15.129:0/2377561527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8260c5bf0 con 0x55a8260b48b0 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f2653fff640 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260c4170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26440b9820 tx=0x7f2644005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d264d525ea3b7749 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.396+0000 7f26337fe640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.397+0000 7f26337fe640 1 -- 172.21.15.129:0/2377561527 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644049050 con 0x55a8260b48b0 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.397+0000 7f265d01e740 1 -- 172.21.15.129:0/2377561527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26440c5a60 con 0x55a8260b48b0 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.397+0000 7f26337fe640 1 -- 172.21.15.129:0/2377561527 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f264403b670 con 0x55a8260b48b0 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.397+0000 7f26337fe640 1 -- 172.21.15.129:0/2377561527 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f26440c11a0 con 0x55a8260b48b0 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.397+0000 7f265d01e740 10 client.25121 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:14.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.397+0000 7f265d01e740 1 -- 172.21.15.129:0/2377561527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a8260c5bf0 con 0x55a8260b48b0 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.397+0000 7f26337fe640 1 -- 172.21.15.129:0/2377561527 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26440c5a60 con 0x55a8260b48b0 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.398+0000 7f26337fe640 1 -- 172.21.15.129:0/2377561527 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f26440bc480 con 0x55a8260b48b0 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.398+0000 7f265d01e740 10 client.25121 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.398+0000 7f265d01e740 10 client.25121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.398+0000 7f265d01e740 20 client.25121 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.398+0000 7f265d01e740 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55a826131810 0x55a826133c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.398+0000 7f265d01e740 4 client.25121 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.398+0000 7f265d01e740 1 -- 172.21.15.129:0/2377561527 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a826027710 con 0x55a826131810 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.398+0000 7f26537fe640 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55a826131810 0x55a826133c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.399+0000 7f26337fe640 10 client.25121 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.399+0000 7f26537fe640 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55a826131810 0x55a826133c00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=326f6e2b9996d21b server_cookie=35083ff30d1c8ef0 in_seq=0 out_seq=0 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f26337fe640 1 -- 172.21.15.129:0/2377561527 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a826027710 con 0x55a826131810 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f26337fe640 10 client.25121 handle_command_reply: tid=0 2024-04-26T21:19:14.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 client.25121 shutdown 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 -- 172.21.15.129:0/2377561527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 msgr2=0x55a8260c4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260c4170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26440b9820 tx=0x7f2644005fd0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 -- 172.21.15.129:0/2377561527 shutdown_connections 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55a826131810 0x55a826133c00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8260ba410 0x55a8260c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a8260b52a0 0x55a8260c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 --2- 172.21.15.129:0/2377561527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8260b48b0 0x55a8260c4170 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f26337fe640 0 client.25121 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 -- 172.21.15.129:0/2377561527 >> 172.21.15.129:0/2377561527 conn(0x55a8260aff20 msgr2=0x55a8260b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 -- 172.21.15.129:0/2377561527 shutdown_connections 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.400+0000 7f265d01e740 1 -- 172.21.15.129:0/2377561527 wait complete. 2024-04-26T21:19:14.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265d01e740 20 client.25121 trim_cache size 0 max 16384 2024-04-26T21:19:14.405 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:19:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:19:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "29fdca0b-2075-43f3-bb20-1f2c8488368e", 2024-04-26T21:19:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:19:14.405 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:19:14.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 -- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f263c0a3cc0 msgr2=0x7f263c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f263c0a3cc0 0x7f263c0a6180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f264c022000 tx=0x7f264c023000 comp rx=0 tx=0).stop 2024-04-26T21:19:14.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 -- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 msgr2=0x7f2654198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 0x7f2654198be0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2644031500 tx=0x7f2644003c10 comp rx=0 tx=0).stop 2024-04-26T21:19:14.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 -- 172.21.15.129:0/644295276 shutdown_connections 2024-04-26T21:19:14.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f263c0a3cc0 0x7f263c0a6180 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2654103420 0x7f26541a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2654102a60 0x7f2654199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 --2- 172.21.15.129:0/644295276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2654101870 0x7f2654198be0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.401+0000 7f265a30a640 1 -- 172.21.15.129:0/644295276 >> 172.21.15.129:0/644295276 conn(0x7f26540fcde0 msgr2=0x7f2654105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.402+0000 7f265a30a640 1 -- 172.21.15.129:0/644295276 shutdown_connections 2024-04-26T21:19:14.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.402+0000 7f265a30a640 1 -- 172.21.15.129:0/644295276 wait complete. 2024-04-26T21:19:14.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:19:14.417 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:19:14.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:14 smithi129 ceph-mon[29237]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.573+0000 7f29a3171640 1 -- 172.21.15.129:0/127447919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f299c1034b0 msgr2=0x7f299c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.573+0000 7f29a3171640 1 --2- 172.21.15.129:0/127447919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f299c1034b0 0x7f299c1058a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2990009900 tx=0x7f299002a410 comp rx=0 tx=0).stop 2024-04-26T21:19:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.574+0000 7f29a3171640 1 -- 172.21.15.129:0/127447919 shutdown_connections 2024-04-26T21:19:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.574+0000 7f29a3171640 1 --2- 172.21.15.129:0/127447919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c105de0 0x7f299c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.574+0000 7f29a3171640 1 --2- 172.21.15.129:0/127447919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c108710 0x7f299c10ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.574+0000 7f29a3171640 1 --2- 172.21.15.129:0/127447919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f299c1034b0 0x7f299c1058a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.574+0000 7f29a3171640 1 -- 172.21.15.129:0/127447919 >> 172.21.15.129:0/127447919 conn(0x7f299c0fcde0 msgr2=0x7f299c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.574+0000 7f29a3171640 1 -- 172.21.15.129:0/127447919 shutdown_connections 2024-04-26T21:19:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.574+0000 7f29a3171640 1 -- 172.21.15.129:0/127447919 wait complete. 2024-04-26T21:19:14.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.574+0000 7f29a3171640 1 Processor -- start 2024-04-26T21:19:14.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.574+0000 7f29a3171640 1 -- start start 2024-04-26T21:19:14.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a3171640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c1034b0 0x7f299c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a3171640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c105de0 0x7f299c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a3171640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f299c108710 0x7f299c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a3171640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f299c10e520 con 0x7f299c108710 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a3171640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f299c10e3a0 con 0x7f299c1034b0 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a3171640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f299c10e6a0 con 0x7f299c105de0 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a0f0d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c1034b0 0x7f299c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a0f0d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c1034b0 0x7f299c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:54008/0 (socket says 172.21.15.129:54008) 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a0f0d640 1 -- 172.21.15.129:0/2932280615 learned_addr learned my addr 172.21.15.129:0/2932280615 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a170e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f299c108710 0x7f299c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f299bfff640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c105de0 0x7f299c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a0f0d640 1 -- 172.21.15.129:0/2932280615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c105de0 msgr2=0x7f299c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c105de0 0x7f299c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a0f0d640 1 -- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f299c108710 msgr2=0x7f299c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f299c108710 0x7f299c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a0f0d640 1 -- 172.21.15.129:0/2932280615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c1a0810 con 0x7f299c1034b0 2024-04-26T21:19:14.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.575+0000 7f29a170e640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f299c108710 0x7f299c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:14.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.576+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c1034b0 0x7f299c198bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2990005ab0 tx=0x7f2990003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.576+0000 7f2999ffb640 1 -- 172.21.15.129:0/2932280615 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299003f070 con 0x7f299c1034b0 2024-04-26T21:19:14.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.576+0000 7f29a3171640 1 -- 172.21.15.129:0/2932280615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c1a0aa0 con 0x7f299c1034b0 2024-04-26T21:19:14.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.576+0000 7f2999ffb640 1 -- 172.21.15.129:0/2932280615 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2990031c50 con 0x7f299c1034b0 2024-04-26T21:19:14.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.576+0000 7f2999ffb640 1 -- 172.21.15.129:0/2932280615 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990035030 con 0x7f299c1034b0 2024-04-26T21:19:14.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.577+0000 7f29a3171640 1 -- 172.21.15.129:0/2932280615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c1a0f90 con 0x7f299c1034b0 2024-04-26T21:19:14.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f2999ffb640 1 -- 172.21.15.129:0/2932280615 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2990042070 con 0x7f299c1034b0 2024-04-26T21:19:14.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f29a5f64740 1 Processor -- start 2024-04-26T21:19:14.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f29a5f64740 1 -- start start 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b102f7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f7620 0x561b102f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 0x561b102fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f29a5f64740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561b10251d80 con 0x561b102f7620 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f29a5f64740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b1020b8b0 con 0x561b102f60a0 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f29a5f64740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561b1024dd40 con 0x561b102f6c30 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.580+0000 7f29a0f0d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f7620 0x561b102f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a0f0d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f7620 0x561b102f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58250/0 (socket says 172.21.15.129:58250) 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a0f0d640 1 -- 172.21.15.129:0/3086692374 learned_addr learned my addr 172.21.15.129:0/3086692374 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a170e640 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b102f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f299bfff640 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 0x561b102fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f2999ffb640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f298c0a3c40 0x7f298c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a170e640 1 -- 172.21.15.129:0/3086692374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 msgr2=0x561b102fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a170e640 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 0x561b102fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a170e640 1 -- 172.21.15.129:0/3086692374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f7620 msgr2=0x561b102f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a170e640 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f7620 0x561b102f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a170e640 1 -- 172.21.15.129:0/3086692374 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b101dca40 con 0x561b102f6c30 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f7620 0x561b102f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f2999ffb640 1 -- 172.21.15.129:0/2932280615 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f29900f0d70 con 0x7f299c1034b0 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a170e640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f298c0a3c40 0x7f298c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.581+0000 7f29a170e640 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b102f7050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2994010690 tx=0x7f2994010b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb4afab2222aae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.582+0000 7f29a170e640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f298c0a3c40 0x7f298c0a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2994021000 tx=0x7f2994024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.582+0000 7f2986ffd640 1 -- 172.21.15.129:0/3086692374 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994024070 con 0x561b102f6c30 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.582+0000 7f2986ffd640 1 -- 172.21.15.129:0/3086692374 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2994012ca0 con 0x561b102f6c30 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.582+0000 7f2986ffd640 1 -- 172.21.15.129:0/3086692374 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299401d3b0 con 0x561b102f6c30 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 -- 172.21.15.129:0/3086692374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 msgr2=0x561b102f7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b102f7050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2994010690 tx=0x7f2994010b50 comp rx=0 tx=0).stop 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 -- 172.21.15.129:0/3086692374 shutdown_connections 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 0x561b102fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f7620 0x561b102f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3086692374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b102f7050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 -- 172.21.15.129:0/3086692374 >> 172.21.15.129:0/3086692374 conn(0x561b102e8350 msgr2=0x561b102e8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 -- 172.21.15.129:0/3086692374 shutdown_connections 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 -- 172.21.15.129:0/3086692374 wait complete. 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 -- start start 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.583+0000 7f29a5f64740 1 Processor -- start 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 0x561b102ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b10300130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10307550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a5f64740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561b102fe2b0 con 0x561b10305160 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a5f64740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b101d8d10 con 0x561b102f60a0 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a5f64740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561b102fe430 con 0x561b102f6c30 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a5f64740 10 client.0 _ll_register_callbacks cb 0x561b10205210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a170e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10307550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a0f0d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 0x561b102ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a170e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10307550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58266/0 (socket says 172.21.15.129:58266) 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f29a170e640 1 -- 172.21.15.129:0/3234982340 learned_addr learned my addr 172.21.15.129:0/3234982340 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.584+0000 7f299bfff640 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b10300130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f299bfff640 1 -- 172.21.15.129:0/3234982340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 msgr2=0x561b102ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f299bfff640 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 0x561b102ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f299bfff640 1 -- 172.21.15.129:0/3234982340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 msgr2=0x561b10307550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f299bfff640 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10307550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f299bfff640 1 -- 172.21.15.129:0/3234982340 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b1002d7d0 con 0x561b102f6c30 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 0x561b102ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f29a170e640 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10307550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f299bfff640 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b10300130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f299c19a100 tx=0x7f2988012aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2836e7f638929ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f297bfff640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f297bfff640 1 -- 172.21.15.129:0/3234982340 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988004050 con 0x561b102f6c30 2024-04-26T21:19:14.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f29a5f64740 1 -- 172.21.15.129:0/3234982340 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2994012ca0 con 0x561b102f6c30 2024-04-26T21:19:14.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f297bfff640 1 -- 172.21.15.129:0/3234982340 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f29880041f0 con 0x561b102f6c30 2024-04-26T21:19:14.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f297bfff640 1 -- 172.21.15.129:0/3234982340 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f29880043f0 con 0x561b102f6c30 2024-04-26T21:19:14.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f29a5f64740 10 client.25078 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:14.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.585+0000 7f29a5f64740 1 -- 172.21.15.129:0/3234982340 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561b0ff633f0 con 0x561b102f6c30 2024-04-26T21:19:14.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.586+0000 7f297bfff640 1 -- 172.21.15.129:0/3234982340 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2994012ca0 con 0x561b102f6c30 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.586+0000 7f297bfff640 1 -- 172.21.15.129:0/3234982340 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f298800f7d0 con 0x561b102f6c30 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.586+0000 7f29a5f64740 10 client.25078 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.586+0000 7f29a5f64740 10 client.25078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.586+0000 7f29a5f64740 20 client.25078 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.586+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561b1030bf80 0x561b1030e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.586+0000 7f29a5f64740 4 client.25078 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.586+0000 7f29a5f64740 1 -- 172.21.15.129:0/3234982340 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b1025f990 con 0x561b1030bf80 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.587+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561b1030bf80 0x561b1030e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.587+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561b1030bf80 0x561b1030e3c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb65323d94349c3 server_cookie=1e746f32e28f9ea2 in_seq=0 out_seq=0 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.587+0000 7f297bfff640 10 client.25078 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f297bfff640 1 -- 172.21.15.129:0/3234982340 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b1025f990 con 0x561b1030bf80 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f297bfff640 10 client.25078 handle_command_reply: tid=0 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 client.25078 shutdown 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 -- 172.21.15.129:0/3234982340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 msgr2=0x561b10300130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b10300130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f299c19a100 tx=0x7f2988012aa0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 -- 172.21.15.129:0/3234982340 shutdown_connections 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561b1030bf80 0x561b1030e3c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10307550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f297bfff640 0 client.25078 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f6c30 0x561b10300130 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 --2- 172.21.15.129:0/3234982340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f60a0 0x561b102ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 -- 172.21.15.129:0/3234982340 >> 172.21.15.129:0/3234982340 conn(0x561b102e8350 msgr2=0x561b102f5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 -- 172.21.15.129:0/3234982340 shutdown_connections 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 1 -- 172.21.15.129:0/3234982340 wait complete. 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.588+0000 7f29a5f64740 20 client.25078 trim_cache size 0 max 16384 2024-04-26T21:19:14.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f29a5f64740 1 Processor -- start 2024-04-26T21:19:14.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f29a5f64740 1 -- start start 2024-04-26T21:19:14.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b10307be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f6c30 0x561b102ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b10305160 0x561b10300130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f29a5f64740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1a5c90 con 0x561b102f6c30 2024-04-26T21:19:14.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f29a5f64740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1a8510 con 0x561b10305160 2024-04-26T21:19:14.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f29a5f64740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1a6300 con 0x561b102f60a0 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f299bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f6c30 0x561b102ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f299bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f6c30 0x561b102ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58278/0 (socket says 172.21.15.129:58278) 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f29a0f0d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b10307be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.623+0000 7f299bfff640 1 -- 172.21.15.129:0/2145946019 learned_addr learned my addr 172.21.15.129:0/2145946019 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f29a170e640 1 --2- 172.21.15.129:0/2145946019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b10305160 0x561b10300130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f299bfff640 1 -- 172.21.15.129:0/2145946019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 msgr2=0x561b10307be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f299bfff640 1 --2- 172.21.15.129:0/2145946019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b10307be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f299bfff640 1 -- 172.21.15.129:0/2145946019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b10305160 msgr2=0x561b10300130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f299bfff640 1 --2- 172.21.15.129:0/2145946019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b10305160 0x561b10300130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f299bfff640 1 -- 172.21.15.129:0/2145946019 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0ff230f0 con 0x561b102f6c30 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f299bfff640 1 --2- 172.21.15.129:0/2145946019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f6c30 0x561b102ffbf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f298800fcc0 tx=0x7f298800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2905b90449553488 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f29867fc640 1 -- 172.21.15.129:0/2145946019 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800f900 con 0x561b102f6c30 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f29867fc640 1 -- 172.21.15.129:0/2145946019 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2988005520 con 0x561b102f6c30 2024-04-26T21:19:14.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.624+0000 7f29867fc640 1 -- 172.21.15.129:0/2145946019 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29880048e0 con 0x561b102f6c30 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.625+0000 7f29a5f64740 1 -- 172.21.15.129:0/2145946019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f6c30 msgr2=0x561b102ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.625+0000 7f29a5f64740 1 --2- 172.21.15.129:0/2145946019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f6c30 0x561b102ffbf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f298800fcc0 tx=0x7f298800f710 comp rx=0 tx=0).stop 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.625+0000 7f29a5f64740 1 -- 172.21.15.129:0/2145946019 shutdown_connections 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.625+0000 7f29a5f64740 1 --2- 172.21.15.129:0/2145946019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b10305160 0x561b10300130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.625+0000 7f29a5f64740 1 --2- 172.21.15.129:0/2145946019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b102f6c30 0x561b102ffbf0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.625+0000 7f29a5f64740 1 --2- 172.21.15.129:0/2145946019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b10307be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.625+0000 7f29a5f64740 1 -- 172.21.15.129:0/2145946019 >> 172.21.15.129:0/2145946019 conn(0x561b102e8350 msgr2=0x561b102e8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.626+0000 7f29a5f64740 1 -- 172.21.15.129:0/2145946019 shutdown_connections 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.626+0000 7f29a5f64740 1 -- 172.21.15.129:0/2145946019 wait complete. 2024-04-26T21:19:14.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.626+0000 7f29a5f64740 1 -- start start 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.626+0000 7f29a5f64740 1 Processor -- start 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b1031c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f6c30 0x561b1031cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a5f64740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10370010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a5f64740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1a69d0 con 0x561b10305160 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a5f64740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1a7e40 con 0x561b102f6c30 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a5f64740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1a70a0 con 0x561b102f60a0 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a0f0d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b1031c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a0f0d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b1031c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:60898/0 (socket says 172.21.15.129:60898) 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a0f0d640 1 -- 172.21.15.129:0/1167352097 learned_addr learned my addr 172.21.15.129:0/1167352097 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f299bfff640 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f6c30 0x561b1031cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a170e640 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10370010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.627+0000 7f29a5f64740 10 client.0 _ll_register_callbacks cb 0x7f29880097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:14.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a0f0d640 1 -- 172.21.15.129:0/1167352097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f6c30 msgr2=0x561b1031cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f6c30 0x561b1031cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a0f0d640 1 -- 172.21.15.129:0/1167352097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 msgr2=0x561b10370010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10370010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a0f0d640 1 -- 172.21.15.129:0/1167352097 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b102124b0 con 0x561b102f60a0 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a170e640 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10370010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a0f0d640 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b1031c570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2990000c00 tx=0x7f29900f48c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459d6d132e538e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f297bfff640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f297bfff640 1 -- 172.21.15.129:0/1167352097 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900d5070 con 0x561b102f60a0 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f297bfff640 1 -- 172.21.15.129:0/1167352097 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f29900bb290 con 0x561b102f60a0 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a5f64740 1 -- 172.21.15.129:0/1167352097 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2988005520 con 0x561b102f60a0 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f297bfff640 1 -- 172.21.15.129:0/1167352097 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2990049040 con 0x561b102f60a0 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a5f64740 10 client.25084 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.628+0000 7f29a5f64740 1 -- 172.21.15.129:0/1167352097 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561b10371450 con 0x561b102f60a0 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.629+0000 7f297bfff640 1 -- 172.21.15.129:0/1167352097 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2990031e00 con 0x561b102f60a0 2024-04-26T21:19:14.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.629+0000 7f297bfff640 1 -- 172.21.15.129:0/1167352097 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f29900b9600 con 0x561b102f60a0 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.629+0000 7f29a5f64740 10 client.25084 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.629+0000 7f29a5f64740 10 client.25084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.629+0000 7f29a5f64740 20 client.25084 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.629+0000 7f29a5f64740 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561b10371db0 0x561b103741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.629+0000 7f29a5f64740 4 client.25084 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.629+0000 7f29a5f64740 1 -- 172.21.15.129:0/1167352097 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2988005520 con 0x561b10371db0 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.629+0000 7f299bfff640 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561b10371db0 0x561b103741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f297bfff640 10 client.25084 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f299bfff640 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561b10371db0 0x561b103741d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3280ffb51c24406c server_cookie=39b377f50443ca4e in_seq=0 out_seq=0 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f297bfff640 1 -- 172.21.15.129:0/1167352097 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f2988005520 con 0x561b10371db0 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f297bfff640 10 client.25084 handle_command_reply: tid=0 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f29a5f64740 1 client.25084 shutdown 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f29a5f64740 1 -- 172.21.15.129:0/1167352097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 msgr2=0x561b1031c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f29a5f64740 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b1031c570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2990000c00 tx=0x7f29900f48c0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f29a5f64740 1 -- 172.21.15.129:0/1167352097 shutdown_connections 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f29a5f64740 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561b10371db0 0x561b103741d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f29a5f64740 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b102f6c30 0x561b1031cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f29a5f64740 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b10305160 0x561b10370010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f297bfff640 0 client.25084 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f29a5f64740 1 --2- 172.21.15.129:0/1167352097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b102f60a0 0x561b1031c570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.630+0000 7f29a5f64740 1 -- 172.21.15.129:0/1167352097 >> 172.21.15.129:0/1167352097 conn(0x561b102e8350 msgr2=0x561b102ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.631+0000 7f29a5f64740 1 -- 172.21.15.129:0/1167352097 shutdown_connections 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.631+0000 7f29a5f64740 1 -- 172.21.15.129:0/1167352097 wait complete. 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.631+0000 7f29a5f64740 20 client.25084 trim_cache size 0 max 16384 2024-04-26T21:19:14.636 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:19:14.637 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:19:14.637 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:19:14.637 INFO:teuthology.orchestra.run.smithi129.stdout: "29fdca0b-2075-43f3-bb20-1f2c8488368e": { 2024-04-26T21:19:14.637 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:19:14.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "29fdca0b-2075-43f3-bb20-1f2c8488368e", 2024-04-26T21:19:14.637 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:19:14.637 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:19:14.637 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:19:14.637 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:19:14.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.631+0000 7f29a3171640 1 -- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f298c0a3c40 msgr2=0x7f298c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.631+0000 7f29a3171640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f298c0a3c40 0x7f298c0a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2994021000 tx=0x7f2994024000 comp rx=0 tx=0).stop 2024-04-26T21:19:14.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.631+0000 7f29a3171640 1 -- 172.21.15.129:0/2932280615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c1034b0 msgr2=0x7f299c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.631+0000 7f29a3171640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c1034b0 0x7f299c198bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2990005ab0 tx=0x7f2990003b90 comp rx=0 tx=0).stop 2024-04-26T21:19:14.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.632+0000 7f29a3171640 1 -- 172.21.15.129:0/2932280615 shutdown_connections 2024-04-26T21:19:14.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.632+0000 7f29a3171640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f298c0a3c40 0x7f298c0a6100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.632+0000 7f29a3171640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f299c108710 0x7f299c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.632+0000 7f29a3171640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c105de0 0x7f299c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.632+0000 7f29a3171640 1 --2- 172.21.15.129:0/2932280615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c1034b0 0x7f299c198bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.632+0000 7f29a3171640 1 -- 172.21.15.129:0/2932280615 >> 172.21.15.129:0/2932280615 conn(0x7f299c0fcde0 msgr2=0x7f299c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.632+0000 7f29a3171640 1 -- 172.21.15.129:0/2932280615 shutdown_connections 2024-04-26T21:19:14.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:14.632+0000 7f29a3171640 1 -- 172.21.15.129:0/2932280615 wait complete. 2024-04-26T21:19:14.648 INFO:tasks.cephfs.filesystem:scrub status for tag:29fdca0b-2075-43f3-bb20-1f2c8488368e - {'path': '/', 'tag': '29fdca0b-2075-43f3-bb20-1f2c8488368e', 'options': 'recursive,force'} 2024-04-26T21:19:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:14 smithi171 ceph-mon[33478]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:14.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:14 smithi150 ceph-mon[32290]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:15 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:19:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:15 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:15 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:19:15.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:15 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:15 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:19:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:15 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:15 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:19:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:15 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:15 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:19:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:15 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:15 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:19:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:15 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:16.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:16 smithi129 ceph-mon[29237]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:16 smithi171 ceph-mon[33478]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:16.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:16 smithi150 ceph-mon[32290]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:17.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:17 smithi129 ceph-mon[29237]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:19:17.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:17 smithi171 ceph-mon[33478]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:19:17.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:17 smithi150 ceph-mon[32290]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:19:19.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:19 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:19:19.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:19 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:19:19.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:19 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:19:20.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:20 smithi150 ceph-mon[32290]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:19:20.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:20 smithi129 ceph-mon[29237]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:19:20.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:20 smithi171 ceph-mon[33478]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:19:22.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:22 smithi150 ceph-mon[32290]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:22.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:22 smithi129 ceph-mon[29237]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:22.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:22 smithi171 ceph-mon[33478]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:23 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:19:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:23 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:19:23.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:23 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:19:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:24 smithi129 ceph-mon[29237]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:19:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:24 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:24 smithi171 ceph-mon[33478]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:19:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:24 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:24 smithi150 ceph-mon[32290]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:19:24.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:24.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:24 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:19:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:19:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:25.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:25 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:19:25.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:25 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:19:25.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:25 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:25.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:25 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:19:25.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:25 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:19:25.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:25 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:19:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:26 smithi129 ceph-mon[29237]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:26 smithi171 ceph-mon[33478]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:26.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:26 smithi150 ceph-mon[32290]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:28.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:28 smithi129 ceph-mon[29237]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:28 smithi171 ceph-mon[33478]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:28.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:28 smithi150 ceph-mon[32290]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:30.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:30 smithi129 ceph-mon[29237]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:19:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:30 smithi171 ceph-mon[33478]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:19:30.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:30 smithi150 ceph-mon[32290]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:19:31.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:31 smithi129 ceph-mon[29237]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:31.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:31 smithi171 ceph-mon[33478]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:31.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:31 smithi150 ceph-mon[32290]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:32.342 INFO:tasks.workunit.client.0.smithi129.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr: git switch -c 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr:Or undo this operation with: 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:32.343 INFO:tasks.workunit.client.0.smithi129.stderr: git switch - 2024-04-26T21:19:32.344 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:32.344 INFO:tasks.workunit.client.0.smithi129.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T21:19:32.344 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:32.344 INFO:tasks.workunit.client.0.smithi129.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T21:19:32.349 DEBUG:teuthology.orchestra.run.smithi129:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T21:19:32.372 INFO:tasks.workunit.client.0.smithi129.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T21:19:32.374 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T21:19:32.374 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T21:19:32.938 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T21:19:32.992 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T21:19:33.031 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T21:19:33.033 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T21:19:33.033 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T21:19:33.071 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T21:19:33.074 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:19:33.074 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T21:19:33.130 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-04-26T21:19:33.131 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-04-26T21:19:33.132 DEBUG:teuthology.orchestra.run.smithi129:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-04-26T21:19:33.192 INFO:tasks.workunit.client.0.smithi129.stdout:getting blogbench 2024-04-26T21:19:33.192 INFO:tasks.workunit.client.0.smithi129.stderr:+ echo 'getting blogbench' 2024-04-26T21:19:33.192 INFO:tasks.workunit.client.0.smithi129.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-04-26T21:19:33.211 INFO:tasks.workunit.client.0.smithi129.stderr:--2024-04-26 21:19:33-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-04-26T21:19:33.212 INFO:tasks.workunit.client.0.smithi129.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-26T21:19:33.233 INFO:tasks.workunit.client.0.smithi129.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-26T21:19:33.253 INFO:tasks.workunit.client.0.smithi129.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-26T21:19:33.254 INFO:tasks.workunit.client.0.smithi129.stderr:Length: 103355 (101K) [application/octet-stream] 2024-04-26T21:19:33.255 INFO:tasks.workunit.client.0.smithi129.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-04-26T21:19:33.257 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:33.295 INFO:tasks.workunit.client.0.smithi129.stderr: 0K .......... .......... .......... .......... .......... 49% 1.25M 0s 2024-04-26T21:19:33.315 INFO:tasks.workunit.client.0.smithi129.stderr: 50K .......... .......... .......... .......... .......... 99% 2.41M 0s 2024-04-26T21:19:33.315 INFO:tasks.workunit.client.0.smithi129.stderr: 100K 100% 1.74T=0.06s 2024-04-26T21:19:33.315 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:33.315 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-26 21:19:33 (1.66 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-04-26T21:19:33.315 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-26T21:19:33.315 INFO:tasks.workunit.client.0.smithi129.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-04-26T21:19:33.348 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/ 2024-04-26T21:19:33.349 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/m4/ 2024-04-26T21:19:33.350 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-04-26T21:19:33.352 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/man/ 2024-04-26T21:19:33.353 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/man/blogbench.8 2024-04-26T21:19:33.355 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/man/Makefile.am 2024-04-26T21:19:33.356 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/man/Makefile.in 2024-04-26T21:19:33.357 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/ 2024-04-26T21:19:33.359 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/mysnprintf.c 2024-04-26T21:19:33.360 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/mysnprintf.h 2024-04-26T21:19:33.362 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/blogbench.c 2024-04-26T21:19:33.363 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/blogbench.h 2024-04-26T21:19:33.365 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/fakesnprintf.c 2024-04-26T21:19:33.367 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/fakesnprintf.h 2024-04-26T21:19:33.368 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/blogbench_p.h 2024-04-26T21:19:33.370 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-04-26T21:19:33.371 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-04-26T21:19:33.372 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/Makefile.am 2024-04-26T21:19:33.373 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/Makefile.in 2024-04-26T21:19:33.375 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/process.c 2024-04-26T21:19:33.376 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/globals.h 2024-04-26T21:19:33.376 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/commenter.c 2024-04-26T21:19:33.378 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/rewriter.c 2024-04-26T21:19:33.378 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/reader.c 2024-04-26T21:19:33.379 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/writer.c 2024-04-26T21:19:33.380 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/src/helpers.c 2024-04-26T21:19:33.381 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/NEWS 2024-04-26T21:19:33.382 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/compile 2024-04-26T21:19:33.383 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/depcomp 2024-04-26T21:19:33.384 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/aclocal.m4 2024-04-26T21:19:33.385 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/README 2024-04-26T21:19:33.386 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/configure 2024-04-26T21:19:33.390 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/configure.ac 2024-04-26T21:19:33.391 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/config.guess 2024-04-26T21:19:33.392 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/install-sh 2024-04-26T21:19:33.393 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/config.sub 2024-04-26T21:19:33.394 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/missing 2024-04-26T21:19:33.395 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/mkinstalldirs 2024-04-26T21:19:33.396 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/Makefile.am 2024-04-26T21:19:33.397 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/Makefile.in 2024-04-26T21:19:33.398 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/config.h.in 2024-04-26T21:19:33.399 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/AUTHORS 2024-04-26T21:19:33.402 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/INSTALL 2024-04-26T21:19:33.402 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/ChangeLog 2024-04-26T21:19:33.402 INFO:tasks.workunit.client.0.smithi129.stdout:blogbench-1.0/COPYING 2024-04-26T21:19:33.403 INFO:tasks.workunit.client.0.smithi129.stderr:+ cd blogbench-1.0/ 2024-04-26T21:19:33.404 INFO:tasks.workunit.client.0.smithi129.stdout:making blogbench 2024-04-26T21:19:33.404 INFO:tasks.workunit.client.0.smithi129.stderr:+ echo 'making blogbench' 2024-04-26T21:19:33.404 INFO:tasks.workunit.client.0.smithi129.stderr:+ ./configure 2024-04-26T21:19:33.514 INFO:tasks.workunit.client.0.smithi129.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-26T21:19:34.522 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether build environment is sane... yes 2024-04-26T21:19:34.531 INFO:tasks.workunit.client.0.smithi129.stdout:checking for gawk... gawk 2024-04-26T21:19:34.548 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether make sets $(MAKE)... yes 2024-04-26T21:19:34.576 INFO:tasks.workunit.client.0.smithi129.stdout:checking for gcc... gcc 2024-04-26T21:19:34.621 INFO:tasks.workunit.client.0.smithi129.stdout:checking for C compiler default output file name... a.out 2024-04-26T21:19:34.622 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether the C compiler works... yes 2024-04-26T21:19:34.626 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether we are cross compiling... no 2024-04-26T21:19:34.662 INFO:tasks.workunit.client.0.smithi129.stdout:checking for suffix of executables... 2024-04-26T21:19:34.700 INFO:tasks.workunit.client.0.smithi129.stdout:checking for suffix of object files... o 2024-04-26T21:19:34.711 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:34 smithi129 ceph-mon[29237]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:34.733 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T21:19:34.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:34 smithi171 ceph-mon[33478]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:34.764 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether gcc accepts -g... yes 2024-04-26T21:19:34.831 INFO:tasks.workunit.client.0.smithi129.stdout:checking for gcc option to accept ANSI C... none needed 2024-04-26T21:19:34.878 INFO:tasks.workunit.client.0.smithi129.stdout:checking for style of include used by make... GNU 2024-04-26T21:19:34.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:34 smithi150 ceph-mon[32290]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:34.968 INFO:tasks.workunit.client.0.smithi129.stdout:checking dependency style of gcc... gcc3 2024-04-26T21:19:34.968 INFO:tasks.workunit.client.0.smithi129.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-26T21:19:35.016 INFO:tasks.workunit.client.0.smithi129.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T21:19:35.055 INFO:tasks.workunit.client.0.smithi129.stdout:checking for egrep... grep -E 2024-04-26T21:19:35.064 INFO:tasks.workunit.client.0.smithi129.stdout:checking for AIX... no 2024-04-26T21:19:35.128 INFO:tasks.workunit.client.0.smithi129.stdout:checking for library containing strerror... none required 2024-04-26T21:19:35.268 INFO:tasks.workunit.client.0.smithi129.stdout:checking for ANSI C header files... yes 2024-04-26T21:19:35.303 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-26T21:19:35.348 INFO:tasks.workunit.client.0.smithi129.stdout:checking for sys/types.h... yes 2024-04-26T21:19:35.396 INFO:tasks.workunit.client.0.smithi129.stdout:checking for sys/stat.h... yes 2024-04-26T21:19:35.440 INFO:tasks.workunit.client.0.smithi129.stdout:checking for stdlib.h... yes 2024-04-26T21:19:35.492 INFO:tasks.workunit.client.0.smithi129.stdout:checking for string.h... yes 2024-04-26T21:19:35.546 INFO:tasks.workunit.client.0.smithi129.stdout:checking for memory.h... yes 2024-04-26T21:19:35.598 INFO:tasks.workunit.client.0.smithi129.stdout:checking for strings.h... yes 2024-04-26T21:19:35.649 INFO:tasks.workunit.client.0.smithi129.stdout:checking for inttypes.h... yes 2024-04-26T21:19:35.702 INFO:tasks.workunit.client.0.smithi129.stdout:checking for stdint.h... yes 2024-04-26T21:19:35.753 INFO:tasks.workunit.client.0.smithi129.stdout:checking for unistd.h... yes 2024-04-26T21:19:35.765 INFO:tasks.workunit.client.0.smithi129.stdout:checking for memory.h... (cached) yes 2024-04-26T21:19:35.770 INFO:tasks.workunit.client.0.smithi129.stdout:checking for strings.h... (cached) yes 2024-04-26T21:19:35.774 INFO:tasks.workunit.client.0.smithi129.stdout:checking for unistd.h... (cached) yes 2024-04-26T21:19:35.817 INFO:tasks.workunit.client.0.smithi129.stdout:checking locale.h usability... yes 2024-04-26T21:19:35.837 INFO:tasks.workunit.client.0.smithi129.stdout:checking locale.h presence... yes 2024-04-26T21:19:35.838 INFO:tasks.workunit.client.0.smithi129.stdout:checking for locale.h... yes 2024-04-26T21:19:35.853 INFO:tasks.workunit.client.0.smithi129.stdout:checking for stdlib.h... (cached) yes 2024-04-26T21:19:35.858 INFO:tasks.workunit.client.0.smithi129.stdout:checking for string.h... (cached) yes 2024-04-26T21:19:35.904 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/socket.h usability... yes 2024-04-26T21:19:35.925 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/socket.h presence... yes 2024-04-26T21:19:35.926 INFO:tasks.workunit.client.0.smithi129.stdout:checking for sys/socket.h... yes 2024-04-26T21:19:36.036 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/time.h usability... yes 2024-04-26T21:19:36.058 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/time.h presence... yes 2024-04-26T21:19:36.058 INFO:tasks.workunit.client.0.smithi129.stdout:checking for sys/time.h... yes 2024-04-26T21:19:36.109 INFO:tasks.workunit.client.0.smithi129.stdout:checking fcntl.h usability... yes 2024-04-26T21:19:36.129 INFO:tasks.workunit.client.0.smithi129.stdout:checking fcntl.h presence... yes 2024-04-26T21:19:36.130 INFO:tasks.workunit.client.0.smithi129.stdout:checking for fcntl.h... yes 2024-04-26T21:19:36.182 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/fcntl.h usability... yes 2024-04-26T21:19:36.202 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/fcntl.h presence... yes 2024-04-26T21:19:36.202 INFO:tasks.workunit.client.0.smithi129.stdout:checking for sys/fcntl.h... yes 2024-04-26T21:19:36.265 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/param.h usability... yes 2024-04-26T21:19:36.286 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/param.h presence... yes 2024-04-26T21:19:36.286 INFO:tasks.workunit.client.0.smithi129.stdout:checking for sys/param.h... yes 2024-04-26T21:19:36.332 INFO:tasks.workunit.client.0.smithi129.stdout:checking ioctl.h usability... no 2024-04-26T21:19:36.352 INFO:tasks.workunit.client.0.smithi129.stdout:checking ioctl.h presence... no 2024-04-26T21:19:36.353 INFO:tasks.workunit.client.0.smithi129.stdout:checking for ioctl.h... no 2024-04-26T21:19:36.402 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/ioctl.h usability... yes 2024-04-26T21:19:36.422 INFO:tasks.workunit.client.0.smithi129.stdout:checking sys/ioctl.h presence... yes 2024-04-26T21:19:36.423 INFO:tasks.workunit.client.0.smithi129.stdout:checking for sys/ioctl.h... yes 2024-04-26T21:19:36.478 INFO:tasks.workunit.client.0.smithi129.stdout:checking return type of signal handlers... void 2024-04-26T21:19:36.521 INFO:tasks.workunit.client.0.smithi129.stdout:checking for size_t... yes 2024-04-26T21:19:36.559 INFO:tasks.workunit.client.0.smithi129.stdout:checking for dev_t... yes 2024-04-26T21:19:36.602 INFO:tasks.workunit.client.0.smithi129.stdout:checking for short... yes 2024-04-26T21:19:36.693 INFO:tasks.workunit.client.0.smithi129.stdout:checking size of short... 2 2024-04-26T21:19:36.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:36 smithi129 ceph-mon[29237]: pgmap v398: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:36.738 INFO:tasks.workunit.client.0.smithi129.stdout:checking for int... yes 2024-04-26T21:19:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:36 smithi171 ceph-mon[33478]: pgmap v398: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:36.805 INFO:tasks.workunit.client.0.smithi129.stdout:checking size of int... 4 2024-04-26T21:19:36.849 INFO:tasks.workunit.client.0.smithi129.stdout:checking for long... yes 2024-04-26T21:19:36.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:36 smithi150 ceph-mon[32290]: pgmap v398: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:36.917 INFO:tasks.workunit.client.0.smithi129.stdout:checking size of long... 8 2024-04-26T21:19:36.961 INFO:tasks.workunit.client.0.smithi129.stdout:checking for long long... yes 2024-04-26T21:19:37.027 INFO:tasks.workunit.client.0.smithi129.stdout:checking size of long long... 8 2024-04-26T21:19:37.068 INFO:tasks.workunit.client.0.smithi129.stdout:checking for ssize_t... yes 2024-04-26T21:19:37.107 INFO:tasks.workunit.client.0.smithi129.stdout:checking for sig_atomic_t... yes 2024-04-26T21:19:37.154 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether gcc needs -traditional... no 2024-04-26T21:19:37.188 INFO:tasks.workunit.client.0.smithi129.stdout:checking for an ANSI C-conforming const... yes 2024-04-26T21:19:37.211 INFO:tasks.workunit.client.0.smithi129.stdout:checking for inline... inline 2024-04-26T21:19:37.270 INFO:tasks.workunit.client.0.smithi129.stdout:checking for working alloca.h... yes 2024-04-26T21:19:37.315 INFO:tasks.workunit.client.0.smithi129.stdout:checking for alloca... yes 2024-04-26T21:19:37.380 INFO:tasks.workunit.client.0.smithi129.stdout:checking for working memcmp... yes 2024-04-26T21:19:37.434 INFO:tasks.workunit.client.0.smithi129.stdout:checking for strftime... yes 2024-04-26T21:19:37.505 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-04-26T21:19:37.563 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether stat accepts an empty string... no 2024-04-26T21:19:37.617 INFO:tasks.workunit.client.0.smithi129.stdout:checking for vprintf... yes 2024-04-26T21:19:37.672 INFO:tasks.workunit.client.0.smithi129.stdout:checking for _doprnt... no 2024-04-26T21:19:37.692 INFO:tasks.workunit.client.0.smithi129.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-26T21:19:37.703 INFO:tasks.workunit.client.0.smithi129.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-26T21:19:37.757 INFO:tasks.workunit.client.0.smithi129.stdout:checking for the pthreads library -lpthreads... no 2024-04-26T21:19:37.815 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether pthreads work without any flags... yes 2024-04-26T21:19:37.862 INFO:tasks.workunit.client.0.smithi129.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-04-26T21:19:37.862 INFO:tasks.workunit.client.0.smithi129.stdout:checking if more special flags are required for pthreads... no 2024-04-26T21:19:37.862 INFO:tasks.workunit.client.0.smithi129.stdout:checking for cc_r... gcc 2024-04-26T21:19:37.935 INFO:tasks.workunit.client.0.smithi129.stdout:checking for setrlimit... yes 2024-04-26T21:19:37.998 INFO:tasks.workunit.client.0.smithi129.stdout:checking for setproctitle... no 2024-04-26T21:19:38.056 INFO:tasks.workunit.client.0.smithi129.stdout:checking for getopt_long... yes 2024-04-26T21:19:38.115 INFO:tasks.workunit.client.0.smithi129.stdout:checking for strtoull... yes 2024-04-26T21:19:38.175 INFO:tasks.workunit.client.0.smithi129.stdout:checking for strtoq... yes 2024-04-26T21:19:38.234 INFO:tasks.workunit.client.0.smithi129.stdout:checking for snprintf... yes 2024-04-26T21:19:38.295 INFO:tasks.workunit.client.0.smithi129.stdout:checking for vsnprintf... yes 2024-04-26T21:19:38.354 INFO:tasks.workunit.client.0.smithi129.stdout:checking for vfprintf... yes 2024-04-26T21:19:38.413 INFO:tasks.workunit.client.0.smithi129.stdout:checking for gethostname... yes 2024-04-26T21:19:38.472 INFO:tasks.workunit.client.0.smithi129.stdout:checking for setlocale... yes 2024-04-26T21:19:38.576 INFO:tasks.workunit.client.0.smithi129.stdout:checking for sync... yes 2024-04-26T21:19:38.646 INFO:tasks.workunit.client.0.smithi129.stdout:checking whether snprintf is C99 conformant... done 2024-04-26T21:19:38.656 INFO:tasks.workunit.client.0.smithi129.stdout:configure: creating ./config.status 2024-04-26T21:19:38.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:38 smithi150 ceph-mon[32290]: pgmap v399: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 938 B/s wr, 2 op/s 2024-04-26T21:19:38.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:38 smithi129 ceph-mon[29237]: pgmap v399: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 938 B/s wr, 2 op/s 2024-04-26T21:19:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:38 smithi171 ceph-mon[33478]: pgmap v399: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 938 B/s wr, 2 op/s 2024-04-26T21:19:38.865 INFO:tasks.workunit.client.0.smithi129.stdout:config.status: creating Makefile 2024-04-26T21:19:38.880 INFO:tasks.workunit.client.0.smithi129.stdout:config.status: creating src/Makefile 2024-04-26T21:19:38.898 INFO:tasks.workunit.client.0.smithi129.stdout:config.status: creating man/Makefile 2024-04-26T21:19:38.910 INFO:tasks.workunit.client.0.smithi129.stdout:config.status: creating config.h 2024-04-26T21:19:38.960 INFO:tasks.workunit.client.0.smithi129.stdout:config.status: executing depfiles commands 2024-04-26T21:19:39.088 INFO:tasks.workunit.client.0.smithi129.stderr:+ make 2024-04-26T21:19:39.098 INFO:tasks.workunit.client.0.smithi129.stdout:make all-recursive 2024-04-26T21:19:39.101 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-26T21:19:39.103 INFO:tasks.workunit.client.0.smithi129.stdout:Making all in src 2024-04-26T21:19:39.109 INFO:tasks.workunit.client.0.smithi129.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-04-26T21:19:39.109 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-04-26T21:19:39.109 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-04-26T21:19:39.192 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-04-26T21:19:39.192 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-04-26T21:19:39.256 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-04-26T21:19:39.256 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-04-26T21:19:39.308 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-04-26T21:19:39.308 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-04-26T21:19:39.358 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-04-26T21:19:39.358 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-04-26T21:19:39.407 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-04-26T21:19:39.407 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-04-26T21:19:39.454 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-04-26T21:19:39.454 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-04-26T21:19:39.525 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-04-26T21:19:39.526 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-04-26T21:19:39.558 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-04-26T21:19:39.559 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-04-26T21:19:39.593 INFO:tasks.workunit.client.0.smithi129.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-04-26T21:19:39.593 INFO:tasks.workunit.client.0.smithi129.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-04-26T21:19:39.628 INFO:tasks.workunit.client.0.smithi129.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-04-26T21:19:39.646 INFO:tasks.workunit.client.0.smithi129.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-04-26T21:19:39.647 INFO:tasks.workunit.client.0.smithi129.stdout:Making all in man 2024-04-26T21:19:39.670 INFO:tasks.workunit.client.0.smithi129.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-04-26T21:19:39.670 INFO:tasks.workunit.client.0.smithi129.stdout:make[2]: Nothing to be done for 'all'. 2024-04-26T21:19:39.670 INFO:tasks.workunit.client.0.smithi129.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-04-26T21:19:39.671 INFO:tasks.workunit.client.0.smithi129.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-26T21:19:39.672 INFO:tasks.workunit.client.0.smithi129.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-26T21:19:39.672 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-26T21:19:39.673 INFO:tasks.workunit.client.0.smithi129.stderr:+ cd src 2024-04-26T21:19:39.673 INFO:tasks.workunit.client.0.smithi129.stderr:+ mkdir blogtest_in 2024-04-26T21:19:39.675 INFO:tasks.workunit.client.0.smithi129.stdout:running blogbench 2024-04-26T21:19:39.675 INFO:tasks.workunit.client.0.smithi129.stderr:+ echo 'running blogbench' 2024-04-26T21:19:39.675 INFO:tasks.workunit.client.0.smithi129.stderr:+ ./blogbench -d blogtest_in 2024-04-26T21:19:39.676 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-26T21:19:39.676 INFO:tasks.workunit.client.0.smithi129.stdout:Frequency = 10 secs 2024-04-26T21:19:39.676 INFO:tasks.workunit.client.0.smithi129.stdout:Scratch dir = [blogtest_in] 2024-04-26T21:19:40.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:40 smithi129 ceph-mon[29237]: pgmap v400: 97 pgs: 97 active+clean; 2.8 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 185 KiB/s wr, 10 op/s 2024-04-26T21:19:40.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:40 smithi171 ceph-mon[33478]: pgmap v400: 97 pgs: 97 active+clean; 2.8 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 185 KiB/s wr, 10 op/s 2024-04-26T21:19:40.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:40 smithi150 ceph-mon[32290]: pgmap v400: 97 pgs: 97 active+clean; 2.8 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 185 KiB/s wr, 10 op/s 2024-04-26T21:19:41.764 INFO:tasks.workunit.client.0.smithi129.stdout:Spawning 3 writers... 2024-04-26T21:19:41.766 INFO:tasks.workunit.client.0.smithi129.stdout:Spawning 1 rewriters... 2024-04-26T21:19:41.766 INFO:tasks.workunit.client.0.smithi129.stdout:Spawning 5 commenters... 2024-04-26T21:19:41.766 INFO:tasks.workunit.client.0.smithi129.stdout:Spawning 100 readers... 2024-04-26T21:19:41.767 INFO:tasks.workunit.client.0.smithi129.stdout:Benchmarking for 30 iterations. 2024-04-26T21:19:41.767 INFO:tasks.workunit.client.0.smithi129.stdout:The test will run during 5 minutes. 2024-04-26T21:19:41.767 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-26T21:19:41.767 INFO:tasks.workunit.client.0.smithi129.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-04-26T21:19:42.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:42 smithi129 ceph-mon[29237]: pgmap v401: 97 pgs: 97 active+clean; 2.9 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 226 KiB/s wr, 21 op/s 2024-04-26T21:19:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:42 smithi171 ceph-mon[33478]: pgmap v401: 97 pgs: 97 active+clean; 2.9 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 226 KiB/s wr, 21 op/s 2024-04-26T21:19:42.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:42 smithi150 ceph-mon[32290]: pgmap v401: 97 pgs: 97 active+clean; 2.9 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 226 KiB/s wr, 21 op/s 2024-04-26T21:19:44.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:19:44.649 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:19:44.673 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:44 smithi129 ceph-mon[29237]: pgmap v402: 97 pgs: 97 active+clean; 5.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 509 KiB/s wr, 48 op/s 2024-04-26T21:19:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:44 smithi171 ceph-mon[33478]: pgmap v402: 97 pgs: 97 active+clean; 5.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 509 KiB/s wr, 48 op/s 2024-04-26T21:19:44.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.758+0000 7f0be8a6d640 1 -- 172.21.15.129:0/3458299309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be41033f0 msgr2=0x7f0be41057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.758+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/3458299309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be41033f0 0x7f0be41057e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009900 tx=0x7f0bd402a410 comp rx=0 tx=0).stop 2024-04-26T21:19:44.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.758+0000 7f0be8a6d640 1 -- 172.21.15.129:0/3458299309 shutdown_connections 2024-04-26T21:19:44.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.758+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/3458299309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be4108650 0x7f0be410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.758+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/3458299309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be4105d20 0x7f0be4108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.758+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/3458299309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be41033f0 0x7f0be41057e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.758+0000 7f0be8a6d640 1 -- 172.21.15.129:0/3458299309 >> 172.21.15.129:0/3458299309 conn(0x7f0be40fcde0 msgr2=0x7f0be40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.758+0000 7f0be8a6d640 1 -- 172.21.15.129:0/3458299309 shutdown_connections 2024-04-26T21:19:44.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.758+0000 7f0be8a6d640 1 -- 172.21.15.129:0/3458299309 wait complete. 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be8a6d640 1 Processor -- start 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be8a6d640 1 -- start start 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be8a6d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be41033f0 0x7f0be4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be8a6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be4105d20 0x7f0be41991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be8a6d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be4108650 0x7f0be41a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be8a6d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0be410e4d0 con 0x7f0be4105d20 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be8a6d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0be410e350 con 0x7f0be4108650 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be8a6d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0be410e650 con 0x7f0be41033f0 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be4105d20 0x7f0be41991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be4105d20 0x7f0be41991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56950/0 (socket says 172.21.15.129:56950) 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0bdbfff640 1 -- 172.21.15.129:0/2865964074 learned_addr learned my addr 172.21.15.129:0/2865964074 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be4108650 0x7f0be41a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0be259c640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be41033f0 0x7f0be4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0bdbfff640 1 -- 172.21.15.129:0/2865964074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be41033f0 msgr2=0x7f0be4198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be41033f0 0x7f0be4198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0bdbfff640 1 -- 172.21.15.129:0/2865964074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be4108650 msgr2=0x7f0be41a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be4108650 0x7f0be41a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0bdbfff640 1 -- 172.21.15.129:0/2865964074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be41a0880 con 0x7f0be4105d20 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.759+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be4105d20 0x7f0be41991b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00d740 tx=0x7f0bcc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.760+0000 7f0bdb7fe640 1 -- 172.21.15.129:0/2865964074 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc017070 con 0x7f0be4105d20 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.760+0000 7f0bdb7fe640 1 -- 172.21.15.129:0/2865964074 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc00de50 con 0x7f0be4105d20 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.760+0000 7f0be8a6d640 1 -- 172.21.15.129:0/2865964074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be41a0b70 con 0x7f0be4105d20 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.760+0000 7f0bdb7fe640 1 -- 172.21.15.129:0/2865964074 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004ce0 con 0x7f0be4105d20 2024-04-26T21:19:44.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.760+0000 7f0be8a6d640 1 -- 172.21.15.129:0/2865964074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be41a1070 con 0x7f0be4105d20 2024-04-26T21:19:44.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0beb785740 1 Processor -- start 2024-04-26T21:19:44.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0bdb7fe640 1 -- 172.21.15.129:0/2865964074 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0bcc010460 con 0x7f0be4105d20 2024-04-26T21:19:44.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0beb785740 1 -- start start 2024-04-26T21:19:44.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0beb785740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e9278300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0beb785740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e9279530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0beb785740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 0x5620e9280300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0beb785740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5620e91dcd80 con 0x5620e92790b0 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0beb785740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5620e91968b0 con 0x5620e9279a70 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0beb785740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5620e91d8d40 con 0x5620e9277ee0 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0be259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e9279530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0be259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e9279530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56966/0 (socket says 172.21.15.129:56966) 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0be259c640 1 -- 172.21.15.129:0/3719644120 learned_addr learned my addr 172.21.15.129:0/3719644120 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 0x5620e9280300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e9278300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0bdb7fe640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0bd00a3c40 0x7f0bd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0be259c640 1 -- 172.21.15.129:0/3719644120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 msgr2=0x5620e9278300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0be259c640 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e9278300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0be259c640 1 -- 172.21.15.129:0/3719644120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 msgr2=0x5620e9280300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0be259c640 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 0x5620e9280300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0be259c640 1 -- 172.21.15.129:0/3719644120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620e9167a40 con 0x5620e92790b0 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.763+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 0x5620e9280300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.764+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0bd00a3c40 0x7f0bd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.764+0000 7f0bdb7fe640 1 -- 172.21.15.129:0/2865964074 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0bcc0cffc0 con 0x7f0be4105d20 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.764+0000 7f0be259c640 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e9279530 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0bd402a930 tx=0x7f0bd4032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee64b6487273e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.764+0000 7f0bd8ff9640 1 -- 172.21.15.129:0/3719644120 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd403f070 con 0x5620e92790b0 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.764+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0bd00a3c40 0x7f0bd00a6100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009fd0 tx=0x7f0bdc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.764+0000 7f0bd8ff9640 1 -- 172.21.15.129:0/3719644120 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd4032e50 con 0x5620e92790b0 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.764+0000 7f0bd8ff9640 1 -- 172.21.15.129:0/3719644120 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4035070 con 0x5620e92790b0 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 -- 172.21.15.129:0/3719644120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 msgr2=0x5620e9279530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e9279530 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0bd402a930 tx=0x7f0bd4032ba0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 -- 172.21.15.129:0/3719644120 shutdown_connections 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 0x5620e9280300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e9279530 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 --2- 172.21.15.129:0/3719644120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e9278300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 -- 172.21.15.129:0/3719644120 >> 172.21.15.129:0/3719644120 conn(0x5620e92733d0 msgr2=0x5620e92737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 -- 172.21.15.129:0/3719644120 shutdown_connections 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 -- 172.21.15.129:0/3719644120 wait complete. 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 -- start start 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.765+0000 7f0beb785740 1 Processor -- start 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0beb785740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9277ee0 0x5620e9282360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0beb785740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92828a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0beb785740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9279a70 0x5620e92898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0beb785740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5620e9280a20 con 0x5620e92790b0 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0beb785740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5620e9163d10 con 0x5620e9277ee0 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0beb785740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5620e9280ba0 con 0x5620e9279a70 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0beb785740 10 client.0 _ll_register_callbacks cb 0x5620e9190210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0be2d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9279a70 0x5620e92898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0be2d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9279a70 0x5620e92898e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:33414/0 (socket says 172.21.15.129:33414) 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0be2d9d640 1 -- 172.21.15.129:0/1452121951 learned_addr learned my addr 172.21.15.129:0/1452121951 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92828a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0be2d9d640 1 -- 172.21.15.129:0/1452121951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9277ee0 msgr2=0x5620e9282360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9277ee0 0x5620e9282360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0be2d9d640 1 -- 172.21.15.129:0/1452121951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 msgr2=0x5620e92828a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.766+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92828a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0be2d9d640 1 -- 172.21.15.129:0/1452121951 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620e8fb87d0 con 0x5620e9279a70 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92828a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9279a70 0x5620e92898e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00fb90 tx=0x7f0bdc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c60775150f1f34a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/1452121951 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01eac0 con 0x5620e9279a70 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0beb785740 1 -- 172.21.15.129:0/1452121951 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd4032e50 con 0x5620e9279a70 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/1452121951 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0bdc01ec60 con 0x5620e9279a70 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/1452121951 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0bdc01ee60 con 0x5620e9279a70 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0beb785740 10 client.25096 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:44.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.767+0000 7f0beb785740 1 -- 172.21.15.129:0/1452121951 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5620e8eee3f0 con 0x5620e9279a70 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/1452121951 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd4032e50 con 0x5620e9279a70 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/1452121951 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f0bdc031050 con 0x5620e9279a70 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0beb785740 10 client.25096 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0beb785740 10 client.25096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0beb785740 20 client.25096 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0beb785740 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5620e928de40 0x5620e9290300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0beb785740 4 client.25096 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0beb785740 1 -- 172.21.15.129:0/1452121951 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620e91ea990 con 0x5620e928de40 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0be259c640 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5620e928de40 0x5620e9290300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0be259c640 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5620e928de40 0x5620e9290300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614f6c2fc553404a server_cookie=876364eeff63d081 in_seq=0 out_seq=0 2024-04-26T21:19:44.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.768+0000 7f0bc9ffb640 10 client.25096 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:44.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.772+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/1452121951 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620e91ea990 con 0x5620e928de40 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.772+0000 7f0bc9ffb640 10 client.25096 handle_command_reply: tid=0 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.772+0000 7f0beb785740 1 client.25096 shutdown 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 -- 172.21.15.129:0/1452121951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9279a70 msgr2=0x5620e92898e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9279a70 0x5620e92898e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00fb90 tx=0x7f0bdc004ab0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 -- 172.21.15.129:0/1452121951 shutdown_connections 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5620e928de40 0x5620e9290300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9279a70 0x5620e92898e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92828a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0bc9ffb640 0 client.25096 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 --2- 172.21.15.129:0/1452121951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9277ee0 0x5620e9282360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 -- 172.21.15.129:0/1452121951 >> 172.21.15.129:0/1452121951 conn(0x5620e92733d0 msgr2=0x5620e9272550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 -- 172.21.15.129:0/1452121951 shutdown_connections 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 1 -- 172.21.15.129:0/1452121951 wait complete. 2024-04-26T21:19:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.773+0000 7f0beb785740 20 client.25096 trim_cache size 0 max 16384 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0beb785740 1 Processor -- start 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0beb785740 1 -- start start 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0beb785740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e926e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0beb785740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e92790b0 0x5620e92762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0beb785740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e9279a70 0x5620e9276820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0beb785740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41a7d60 con 0x5620e9279a70 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0beb785740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41aa5e0 con 0x5620e92790b0 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0beb785740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41a83d0 con 0x5620e9277ee0 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0be2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e9279a70 0x5620e9276820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0be2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e9279a70 0x5620e9276820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56988/0 (socket says 172.21.15.129:56988) 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0be2d9d640 1 -- 172.21.15.129:0/2829858217 learned_addr learned my addr 172.21.15.129:0/2829858217 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/2829858217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e92790b0 0x5620e92762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0be259c640 1 --2- 172.21.15.129:0/2829858217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e926e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0be2d9d640 1 -- 172.21.15.129:0/2829858217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 msgr2=0x5620e926e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.795+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/2829858217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e926e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.796+0000 7f0be2d9d640 1 -- 172.21.15.129:0/2829858217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e92790b0 msgr2=0x5620e92762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.796+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/2829858217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e92790b0 0x5620e92762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.796+0000 7f0be2d9d640 1 -- 172.21.15.129:0/2829858217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620e8eae0f0 con 0x5620e9279a70 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.796+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/2829858217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e9279a70 0x5620e9276820 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009540 tx=0x7f0bdc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edaa3a584f13ed01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.796+0000 7f0bcbfff640 1 -- 172.21.15.129:0/2829858217 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01eac0 con 0x5620e9279a70 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.796+0000 7f0bcbfff640 1 -- 172.21.15.129:0/2829858217 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bdc037020 con 0x5620e9279a70 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.796+0000 7f0bcbfff640 1 -- 172.21.15.129:0/2829858217 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc033610 con 0x5620e9279a70 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 -- 172.21.15.129:0/2829858217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e9279a70 msgr2=0x5620e9276820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 --2- 172.21.15.129:0/2829858217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e9279a70 0x5620e9276820 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009540 tx=0x7f0bdc005f90 comp rx=0 tx=0).stop 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 -- 172.21.15.129:0/2829858217 shutdown_connections 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 --2- 172.21.15.129:0/2829858217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e92790b0 0x5620e92762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 --2- 172.21.15.129:0/2829858217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e9279a70 0x5620e9276820 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 --2- 172.21.15.129:0/2829858217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e926e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 -- 172.21.15.129:0/2829858217 >> 172.21.15.129:0/2829858217 conn(0x5620e92733d0 msgr2=0x5620e927cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 -- 172.21.15.129:0/2829858217 shutdown_connections 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 -- 172.21.15.129:0/2829858217 wait complete. 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 -- start start 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 Processor -- start 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e92ef950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92efe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 0x5620e92f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41a8aa0 con 0x5620e92790b0 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41a9f10 con 0x5620e9279a70 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41a9170 con 0x5620e9277ee0 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0beb785740 10 client.0 _ll_register_callbacks cb 0x7f0bdc009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92efe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56990/0 (socket says 172.21.15.129:56990) 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0bdbfff640 1 -- 172.21.15.129:0/3340907508 learned_addr learned my addr 172.21.15.129:0/3340907508 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.797+0000 7f0be2d9d640 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 0x5620e92f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0be259c640 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e92ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bdbfff640 1 -- 172.21.15.129:0/3340907508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 msgr2=0x5620e92ef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e92ef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bdbfff640 1 -- 172.21.15.129:0/3340907508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 msgr2=0x5620e92f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 0x5620e92f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bdbfff640 1 -- 172.21.15.129:0/3340907508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620e919d4b0 con 0x5620e92790b0 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bdbfff640 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92efe90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009610 tx=0x7f0bcc0cde80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953dda164a7cf2a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/3340907508 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc020040 con 0x5620e92790b0 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/3340907508 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0bcc0a0120 con 0x5620e92790b0 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0beb785740 1 -- 172.21.15.129:0/3340907508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bdc037020 con 0x5620e92790b0 2024-04-26T21:19:44.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/3340907508 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0bcc09a310 con 0x5620e92790b0 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0beb785740 10 client.15549 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0beb785740 1 -- 172.21.15.129:0/3340907508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5620e92f86d0 con 0x5620e92790b0 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/3340907508 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bdc037020 con 0x5620e92790b0 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/3340907508 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f0bcc0d20b0 con 0x5620e92790b0 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0beb785740 10 client.15549 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0beb785740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0beb785740 20 client.15549 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0beb785740 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5620e92f9150 0x5620e92fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0beb785740 4 client.15549 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.798+0000 7f0beb785740 1 -- 172.21.15.129:0/3340907508 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620e91e4800 con 0x5620e92f9150 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.799+0000 7f0be259c640 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5620e92f9150 0x5620e92fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.799+0000 7f0be259c640 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5620e92f9150 0x5620e92fb570 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e216acc1f4f8ae7a server_cookie=4ab9d2303046a51f in_seq=0 out_seq=0 2024-04-26T21:19:44.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.799+0000 7f0bc9ffb640 10 client.15549 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.802+0000 7f0bc9ffb640 1 -- 172.21.15.129:0/3340907508 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5620e91e4800 con 0x5620e92f9150 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.802+0000 7f0bc9ffb640 10 client.15549 handle_command_reply: tid=0 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.802+0000 7f0beb785740 1 client.15549 shutdown 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 -- 172.21.15.129:0/3340907508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 msgr2=0x5620e92efe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92efe90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009610 tx=0x7f0bcc0cde80 comp rx=0 tx=0).stop 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 -- 172.21.15.129:0/3340907508 shutdown_connections 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5620e92f9150 0x5620e92fb570 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620e9279a70 0x5620e92f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5620e92790b0 0x5620e92efe90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 --2- 172.21.15.129:0/3340907508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620e9277ee0 0x5620e92ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0bc9ffb640 0 client.15549 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 -- 172.21.15.129:0/3340907508 >> 172.21.15.129:0/3340907508 conn(0x5620e92733d0 msgr2=0x5620e9271cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 -- 172.21.15.129:0/3340907508 shutdown_connections 2024-04-26T21:19:44.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 1 -- 172.21.15.129:0/3340907508 wait complete. 2024-04-26T21:19:44.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0beb785740 20 client.15549 trim_cache size 0 max 16384 2024-04-26T21:19:44.808 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:19:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:19:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:19:44.808 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:19:44.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 -- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0bd00a3c40 msgr2=0x7f0bd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0bd00a3c40 0x7f0bd00a6100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009fd0 tx=0x7f0bdc004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:44.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 -- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be4105d20 msgr2=0x7f0be41991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be4105d20 0x7f0be41991b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00d740 tx=0x7f0bcc00dc00 comp rx=0 tx=0).stop 2024-04-26T21:19:44.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 -- 172.21.15.129:0/2865964074 shutdown_connections 2024-04-26T21:19:44.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f0bd00a3c40 0x7f0bd00a6100 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be4108650 0x7f0be41a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0be4105d20 0x7f0be41991b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 --2- 172.21.15.129:0/2865964074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be41033f0 0x7f0be4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 -- 172.21.15.129:0/2865964074 >> 172.21.15.129:0/2865964074 conn(0x7f0be40fcde0 msgr2=0x7f0be41052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.803+0000 7f0be8a6d640 1 -- 172.21.15.129:0/2865964074 shutdown_connections 2024-04-26T21:19:44.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.804+0000 7f0be8a6d640 1 -- 172.21.15.129:0/2865964074 wait complete. 2024-04-26T21:19:44.818 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:19:44.818 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:44.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:44 smithi150 ceph-mon[32290]: pgmap v402: 97 pgs: 97 active+clean; 5.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 509 KiB/s wr, 48 op/s 2024-04-26T21:19:44.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.962+0000 7f315cec4640 1 -- 172.21.15.129:0/3316318088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158101870 msgr2=0x7f3158101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.962+0000 7f315cec4640 1 --2- 172.21.15.129:0/3316318088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158101870 0x7f3158101c90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3150009900 tx=0x7f315002a410 comp rx=0 tx=0).stop 2024-04-26T21:19:44.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.962+0000 7f315cec4640 1 -- 172.21.15.129:0/3316318088 shutdown_connections 2024-04-26T21:19:44.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.962+0000 7f315cec4640 1 --2- 172.21.15.129:0/3316318088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158103420 0x7f3158109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.962+0000 7f315cec4640 1 --2- 172.21.15.129:0/3316318088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3158102a60 0x7f3158102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.962+0000 7f315cec4640 1 --2- 172.21.15.129:0/3316318088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158101870 0x7f3158101c90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.962+0000 7f315cec4640 1 -- 172.21.15.129:0/3316318088 >> 172.21.15.129:0/3316318088 conn(0x7f31580fcde0 msgr2=0x7f31580ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.962+0000 7f315cec4640 1 -- 172.21.15.129:0/3316318088 shutdown_connections 2024-04-26T21:19:44.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.962+0000 7f315cec4640 1 -- 172.21.15.129:0/3316318088 wait complete. 2024-04-26T21:19:44.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315cec4640 1 Processor -- start 2024-04-26T21:19:44.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315cec4640 1 -- start start 2024-04-26T21:19:44.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315cec4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3158101870 0x7f315806a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315cec4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158102a60 0x7f315819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315cec4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158103420 0x7f31581a22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315cec4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f315810d390 con 0x7f3158103420 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315cec4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f315810d210 con 0x7f3158102a60 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315cec4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f315810d510 con 0x7f3158101870 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3156d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158103420 0x7f31581a22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3155d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158102a60 0x7f315819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3156d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158103420 0x7f31581a22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57018/0 (socket says 172.21.15.129:57018) 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3156d9d640 1 -- 172.21.15.129:0/3878316915 learned_addr learned my addr 172.21.15.129:0/3878316915 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315659c640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3158101870 0x7f315806a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3156d9d640 1 -- 172.21.15.129:0/3878316915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3158101870 msgr2=0x7f315806a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3156d9d640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3158101870 0x7f315806a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3156d9d640 1 -- 172.21.15.129:0/3878316915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158102a60 msgr2=0x7f315819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3156d9d640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158102a60 0x7f315819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3156d9d640 1 -- 172.21.15.129:0/3878316915 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31581a2ad0 con 0x7f3158103420 2024-04-26T21:19:44.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f3156d9d640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158103420 0x7f31581a22e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f314800e9b0 tx=0x7f314800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f31477fe640 1 -- 172.21.15.129:0/3878316915 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314800cd80 con 0x7f3158103420 2024-04-26T21:19:44.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f31477fe640 1 -- 172.21.15.129:0/3878316915 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31480035f0 con 0x7f3158103420 2024-04-26T21:19:44.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31581a2d60 con 0x7f3158103420 2024-04-26T21:19:44.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.963+0000 7f31477fe640 1 -- 172.21.15.129:0/3878316915 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148010690 con 0x7f3158103420 2024-04-26T21:19:44.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.964+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31581a3200 con 0x7f3158103420 2024-04-26T21:19:44.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.964+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3158108cb0 con 0x7f3158103420 2024-04-26T21:19:44.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.966+0000 7f31477fe640 1 -- 172.21.15.129:0/3878316915 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3148004020 con 0x7f3158103420 2024-04-26T21:19:44.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.966+0000 7f31477fe640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31400a3c70 0x7f31400a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.966+0000 7f315659c640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31400a3c70 0x7f31400a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.967+0000 7f31477fe640 1 -- 172.21.15.129:0/3878316915 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f31480d0820 con 0x7f3158103420 2024-04-26T21:19:44.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.967+0000 7f315659c640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31400a3c70 0x7f31400a6130 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3150009230 tx=0x7f31500314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:44.971+0000 7f31477fe640 1 -- 172.21.15.129:0/3878316915 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f314809d090 con 0x7f3158103420 2024-04-26T21:19:45.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.207+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f315802dd20 con 0x7f3158103420 2024-04-26T21:19:45.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.208+0000 7f31477fe640 1 -- 172.21.15.129:0/3878316915 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f314809c990 con 0x7f3158103420 2024-04-26T21:19:45.209 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:19:45.209 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:19:45.210 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31400a3c70 msgr2=0x7f31400a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31400a3c70 0x7f31400a6130 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3150009230 tx=0x7f31500314a0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158103420 msgr2=0x7f31581a22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158103420 0x7f31581a22e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f314800e9b0 tx=0x7f314800ee70 comp rx=0 tx=0).stop 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 shutdown_connections 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f31400a3c70 0x7f31400a6130 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158102a60 0x7f315819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3158103420 0x7f31581a22e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 --2- 172.21.15.129:0/3878316915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3158101870 0x7f315806a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.210+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 >> 172.21.15.129:0/3878316915 conn(0x7f31580fcde0 msgr2=0x7f3158105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.211+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 shutdown_connections 2024-04-26T21:19:45.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.211+0000 7f315cec4640 1 -- 172.21.15.129:0/3878316915 wait complete. 2024-04-26T21:19:45.223 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:19:45.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.325+0000 7fbbac71b640 1 -- 172.21.15.129:0/1794208919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 msgr2=0x7fbba4109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.325+0000 7fbbac71b640 1 --2- 172.21.15.129:0/1794208919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 0x7fbba4109cb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c02a410 comp rx=0 tx=0).stop 2024-04-26T21:19:45.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 -- 172.21.15.129:0/1794208919 shutdown_connections 2024-04-26T21:19:45.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 --2- 172.21.15.129:0/1794208919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbba4102a60 0x7fbba4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 --2- 172.21.15.129:0/1794208919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 0x7fbba4109cb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 --2- 172.21.15.129:0/1794208919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbba4101870 0x7fbba4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 -- 172.21.15.129:0/1794208919 >> 172.21.15.129:0/1794208919 conn(0x7fbba40fcde0 msgr2=0x7fbba40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 -- 172.21.15.129:0/1794208919 shutdown_connections 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 -- 172.21.15.129:0/1794208919 wait complete. 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 Processor -- start 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 -- start start 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbba4101870 0x7fbba406a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbba4102a60 0x7fbba419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 0x7fbba419ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbba410d410 con 0x7fbba4103420 2024-04-26T21:19:45.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbba410d290 con 0x7fbba4102a60 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbac71b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbba410d590 con 0x7fbba4101870 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.326+0000 7fbbaacb8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 0x7fbba419ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbaacb8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 0x7fbba419ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57038/0 (socket says 172.21.15.129:57038) 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbaacb8640 1 -- 172.21.15.129:0/79371048 learned_addr learned my addr 172.21.15.129:0/79371048 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbba4102a60 0x7fbba419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbaa4b7640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbba4101870 0x7fbba406a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbaacb8640 1 -- 172.21.15.129:0/79371048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbba4101870 msgr2=0x7fbba406a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbaacb8640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbba4101870 0x7fbba406a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbaacb8640 1 -- 172.21.15.129:0/79371048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbba4102a60 msgr2=0x7fbba419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbaacb8640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbba4102a60 0x7fbba419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbaacb8640 1 -- 172.21.15.129:0/79371048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba41a0860 con 0x7fbba4103420 2024-04-26T21:19:45.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbaacb8640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 0x7fbba419ffa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c031500 tx=0x7fbb9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbba37fe640 1 -- 172.21.15.129:0/79371048 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c03f030 con 0x7fbba4103420 2024-04-26T21:19:45.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbba37fe640 1 -- 172.21.15.129:0/79371048 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c003cb0 con 0x7fbba4103420 2024-04-26T21:19:45.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbac71b640 1 -- 172.21.15.129:0/79371048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba41a0af0 con 0x7fbba4103420 2024-04-26T21:19:45.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbba37fe640 1 -- 172.21.15.129:0/79371048 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c035070 con 0x7fbba4103420 2024-04-26T21:19:45.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.327+0000 7fbbac71b640 1 -- 172.21.15.129:0/79371048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba41a0fe0 con 0x7fbba4103420 2024-04-26T21:19:45.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.329+0000 7fbba37fe640 1 -- 172.21.15.129:0/79371048 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbb9c042030 con 0x7fbba4103420 2024-04-26T21:19:45.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaf432740 1 Processor -- start 2024-04-26T21:19:45.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaf432740 1 -- start start 2024-04-26T21:19:45.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83947820 0x561f83947c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 0x561f83948e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f8394fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaf432740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561f838abd80 con 0x561f839493b0 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaf432740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561f83865a20 con 0x561f83947820 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaf432740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561f838a7d40 con 0x561f839489f0 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbba37fe640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbb8c0a3c40 0x7fbb8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbba9cb6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f8394fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaa4b7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 0x561f83948e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbba9cb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83947820 0x561f83947c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaa4b7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 0x561f83948e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:33492/0 (socket says 172.21.15.129:33492) 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaa4b7640 1 -- 172.21.15.129:0/4253125759 learned_addr learned my addr 172.21.15.129:0/4253125759 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.330+0000 7fbbaacb8640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbb8c0a3c40 0x7fbb8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba37fe640 1 -- 172.21.15.129:0/79371048 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fbb9c0f0b40 con 0x7fbba4103420 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba9cb6640 1 -- 172.21.15.129:0/4253125759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 msgr2=0x561f83948e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/4253125759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 0x561f83948e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba9cb6640 1 -- 172.21.15.129:0/4253125759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 msgr2=0x561f8394fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/4253125759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f8394fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba9cb6640 1 -- 172.21.15.129:0/4253125759 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f83870c30 con 0x561f83947820 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/4253125759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83947820 0x561f83947c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb98005fd0 tx=0x7fbb98009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3338c32d5a496b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbbaacb8640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbb8c0a3c40 0x7fbb8c0a6100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0c53f0 tx=0x7fbb9c0e7dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba0ff9640 1 -- 172.21.15.129:0/4253125759 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98016070 con 0x561f83947820 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba0ff9640 1 -- 172.21.15.129:0/4253125759 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800fe50 con 0x561f83947820 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.331+0000 7fbba0ff9640 1 -- 172.21.15.129:0/4253125759 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98019040 con 0x561f83947820 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 -- 172.21.15.129:0/4253125759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83947820 msgr2=0x561f83947c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 --2- 172.21.15.129:0/4253125759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83947820 0x561f83947c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb98005fd0 tx=0x7fbb98009500 comp rx=0 tx=0).stop 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 -- 172.21.15.129:0/4253125759 shutdown_connections 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 --2- 172.21.15.129:0/4253125759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f8394fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 --2- 172.21.15.129:0/4253125759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 0x561f83948e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 --2- 172.21.15.129:0/4253125759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83947820 0x561f83947c40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 -- 172.21.15.129:0/4253125759 >> 172.21.15.129:0/4253125759 conn(0x561f83888910 msgr2=0x561f83943120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 -- 172.21.15.129:0/4253125759 shutdown_connections 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 -- 172.21.15.129:0/4253125759 wait complete. 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 -- start start 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.332+0000 7fbbaf432740 1 Processor -- start 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 0x561f83951a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839489f0 0x561f83951fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f83959010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbbaf432740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561f839503c0 con 0x561f839493b0 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbbaf432740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561f83832dc0 con 0x561f839489f0 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbbaf432740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561f83950540 con 0x561f83947820 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbbaf432740 10 client.0 _ll_register_callbacks cb 0x561f8384e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f83959010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbbaa4b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839489f0 0x561f83951fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f83959010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57052/0 (socket says 172.21.15.129:57052) 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 -- 172.21.15.129:0/3050173386 learned_addr learned my addr 172.21.15.129:0/3050173386 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 0x561f83951a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 -- 172.21.15.129:0/3050173386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 msgr2=0x561f83951a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 0x561f83951a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 -- 172.21.15.129:0/3050173386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839489f0 msgr2=0x561f83951fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839489f0 0x561f83951fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 -- 172.21.15.129:0/3050173386 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f835bd3f0 con 0x561f839493b0 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbbaa4b7640 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839489f0 0x561f83951fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f83959010 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009370 tx=0x7fbb98009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9f92745a1563eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.333+0000 7fbb89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3050173386 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98014070 con 0x561f839493b0 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3050173386 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fbb9800fc50 con 0x561f839493b0 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaf432740 1 -- 172.21.15.129:0/3050173386 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9800fe50 con 0x561f839493b0 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3050173386 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbb98016070 con 0x561f839493b0 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaf432740 10 client.15567 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaf432740 1 -- 172.21.15.129:0/3050173386 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561f836877d0 con 0x561f839493b0 2024-04-26T21:19:45.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3050173386 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800fe50 con 0x561f839493b0 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3050173386 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fbb9802c020 con 0x561f839493b0 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaf432740 10 client.15567 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaf432740 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaf432740 20 client.15567 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaf432740 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561f8395d730 0x561f8395fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaf432740 4 client.15567 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaf432740 1 -- 172.21.15.129:0/3050173386 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f838b9990 con 0x561f8395d730 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.334+0000 7fbbaa4b7640 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561f8395d730 0x561f8395fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.335+0000 7fbb89ffb640 10 client.15567 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:45.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.335+0000 7fbbaa4b7640 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561f8395d730 0x561f8395fb60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5afb37a35a1823af server_cookie=6af15c976c28003c in_seq=0 out_seq=0 2024-04-26T21:19:45.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3050173386 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f838b9990 con 0x561f8395d730 2024-04-26T21:19:45.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbb89ffb640 10 client.15567 handle_command_reply: tid=0 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 client.15567 shutdown 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 -- 172.21.15.129:0/3050173386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 msgr2=0x561f83959010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f83959010 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009370 tx=0x7fbb98009610 comp rx=0 tx=0).stop 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 -- 172.21.15.129:0/3050173386 shutdown_connections 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561f8395d730 0x561f8395fb60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839489f0 0x561f83951fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbb89ffb640 0 client.15567 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839493b0 0x561f83959010 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 --2- 172.21.15.129:0/3050173386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 0x561f83951a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 -- 172.21.15.129:0/3050173386 >> 172.21.15.129:0/3050173386 conn(0x561f83888910 msgr2=0x561f83942280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 -- 172.21.15.129:0/3050173386 shutdown_connections 2024-04-26T21:19:45.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 1 -- 172.21.15.129:0/3050173386 wait complete. 2024-04-26T21:19:45.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.338+0000 7fbbaf432740 20 client.15567 trim_cache size 0 max 16384 2024-04-26T21:19:45.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.351+0000 7fbbaf432740 1 Processor -- start 2024-04-26T21:19:45.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.351+0000 7fbbaf432740 1 -- start start 2024-04-26T21:19:45.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.351+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 0x561f83965430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.351+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839489f0 0x561f83965970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.351+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 0x561f83958fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.351+0000 7fbbaf432740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41a7e50 con 0x561f839489f0 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.351+0000 7fbbaf432740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41aa6d0 con 0x561f839493b0 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.351+0000 7fbbaf432740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41a84c0 con 0x561f83947820 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbbaa4b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839489f0 0x561f83965970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbbaa4b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839489f0 0x561f83965970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57068/0 (socket says 172.21.15.129:57068) 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbba9cb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 0x561f83958fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbbaa4b7640 1 -- 172.21.15.129:0/1696529696 learned_addr learned my addr 172.21.15.129:0/1696529696 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/1696529696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 0x561f83965430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbbaa4b7640 1 -- 172.21.15.129:0/1696529696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 msgr2=0x561f83965430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbbaa4b7640 1 --2- 172.21.15.129:0/1696529696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 0x561f83965430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbbaa4b7640 1 -- 172.21.15.129:0/1696529696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 msgr2=0x561f83958fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbbaa4b7640 1 --2- 172.21.15.129:0/1696529696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 0x561f83958fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbbaa4b7640 1 -- 172.21.15.129:0/1696529696 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f839473c0 con 0x561f839489f0 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/1696529696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 0x561f83958fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbbaa4b7640 1 --2- 172.21.15.129:0/1696529696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839489f0 0x561f83965970 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbb94006a70 tx=0x7fbb94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9da28e19c0fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbb8bfff640 1 -- 172.21.15.129:0/1696529696 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94020030 con 0x561f839489f0 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbb8bfff640 1 -- 172.21.15.129:0/1696529696 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9401c250 con 0x561f839489f0 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.352+0000 7fbb8bfff640 1 -- 172.21.15.129:0/1696529696 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb940244b0 con 0x561f839489f0 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 -- 172.21.15.129:0/1696529696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839489f0 msgr2=0x561f83965970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 --2- 172.21.15.129:0/1696529696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839489f0 0x561f83965970 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbb94006a70 tx=0x7fbb94008040 comp rx=0 tx=0).stop 2024-04-26T21:19:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 -- 172.21.15.129:0/1696529696 shutdown_connections 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 --2- 172.21.15.129:0/1696529696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 0x561f83958fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 --2- 172.21.15.129:0/1696529696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f839489f0 0x561f83965970 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 --2- 172.21.15.129:0/1696529696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f83947820 0x561f83965430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 -- 172.21.15.129:0/1696529696 >> 172.21.15.129:0/1696529696 conn(0x561f83888910 msgr2=0x561f83946010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 -- 172.21.15.129:0/1696529696 shutdown_connections 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 -- 172.21.15.129:0/1696529696 wait complete. 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 -- start start 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 Processor -- start 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f83947820 0x561f839bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 0x561f839bf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 0x561f839c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41a8b90 con 0x561f83947820 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41aa000 con 0x561f839493b0 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.353+0000 7fbbaf432740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41a9260 con 0x561f839489f0 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbbaf432740 10 client.0 _ll_register_callbacks cb 0x7fbb980105b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f83947820 0x561f839bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f83947820 0x561f839bf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57072/0 (socket says 172.21.15.129:57072) 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 -- 172.21.15.129:0/484132883 learned_addr learned my addr 172.21.15.129:0/484132883 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 0x561f839c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 -- 172.21.15.129:0/484132883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 msgr2=0x561f839bf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 0x561f839bf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 -- 172.21.15.129:0/484132883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 msgr2=0x561f839c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 0x561f839c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 -- 172.21.15.129:0/484132883 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f839c6c50 con 0x561f83947820 2024-04-26T21:19:45.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbba9cb6640 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f83947820 0x561f839bf0e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbb98004360 tx=0x7fbb98009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27eb658eff40b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbb89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:45.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbb89ffb640 1 -- 172.21.15.129:0/484132883 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98014040 con 0x561f83947820 2024-04-26T21:19:45.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbb89ffb640 1 -- 172.21.15.129:0/484132883 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fbb98029040 con 0x561f83947820 2024-04-26T21:19:45.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbbaf432740 1 -- 172.21.15.129:0/484132883 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9401c250 con 0x561f83947820 2024-04-26T21:19:45.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbb89ffb640 1 -- 172.21.15.129:0/484132883 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbb98004c90 con 0x561f83947820 2024-04-26T21:19:45.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbbaf432740 10 client.15573 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.354+0000 7fbbaf432740 1 -- 172.21.15.129:0/484132883 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561f839c7f80 con 0x561f83947820 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbb89ffb640 1 -- 172.21.15.129:0/484132883 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9401c250 con 0x561f83947820 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbb89ffb640 1 -- 172.21.15.129:0/484132883 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fbb9802c040 con 0x561f83947820 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbbaf432740 10 client.15573 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbbaf432740 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbbaf432740 20 client.15573 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbbaf432740 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561f839c8950 0x561f839cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbbaf432740 4 client.15573 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbbaf432740 1 -- 172.21.15.129:0/484132883 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f838b3800 con 0x561f839c8950 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbbaa4b7640 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561f839c8950 0x561f839cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbbaa4b7640 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561f839c8950 0x561f839cad70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9236d25dd1814ab2 server_cookie=d3b3717384f04f87 in_seq=0 out_seq=0 2024-04-26T21:19:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.355+0000 7fbb89ffb640 10 client.15573 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:45.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbb89ffb640 1 -- 172.21.15.129:0/484132883 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f838b3800 con 0x561f839c8950 2024-04-26T21:19:45.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbb89ffb640 10 client.15573 handle_command_reply: tid=0 2024-04-26T21:19:45.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 client.15573 shutdown 2024-04-26T21:19:45.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 -- 172.21.15.129:0/484132883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f83947820 msgr2=0x561f839bf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f83947820 0x561f839bf0e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbb98004360 tx=0x7fbb98009990 comp rx=0 tx=0).stop 2024-04-26T21:19:45.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 -- 172.21.15.129:0/484132883 shutdown_connections 2024-04-26T21:19:45.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561f839c8950 0x561f839cad70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f839493b0 0x561f839c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561f839489f0 0x561f839bf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 --2- 172.21.15.129:0/484132883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f83947820 0x561f839bf0e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbb89ffb640 0 client.15573 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:45.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 -- 172.21.15.129:0/484132883 >> 172.21.15.129:0/484132883 conn(0x561f83888910 msgr2=0x561f839468a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 -- 172.21.15.129:0/484132883 shutdown_connections 2024-04-26T21:19:45.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 1 -- 172.21.15.129:0/484132883 wait complete. 2024-04-26T21:19:45.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.359+0000 7fbbaf432740 20 client.15573 trim_cache size 0 max 16384 2024-04-26T21:19:45.366 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:19:45.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 -- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbb8c0a3c40 msgr2=0x7fbb8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbb8c0a3c40 0x7fbb8c0a6100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0c53f0 tx=0x7fbb9c0e7dc0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 -- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 msgr2=0x7fbba419ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 0x7fbba419ffa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c031500 tx=0x7fbb9c0042f0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 -- 172.21.15.129:0/79371048 shutdown_connections 2024-04-26T21:19:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbb8c0a3c40 0x7fbb8c0a6100 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbba4102a60 0x7fbba419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba4103420 0x7fbba419ffa0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 --2- 172.21.15.129:0/79371048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbba4101870 0x7fbba406a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 -- 172.21.15.129:0/79371048 >> 172.21.15.129:0/79371048 conn(0x7fbba40fcde0 msgr2=0x7fbba4105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 -- 172.21.15.129:0/79371048 shutdown_connections 2024-04-26T21:19:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:45.360+0000 7fbbac71b640 1 -- 172.21.15.129:0/79371048 wait complete. 2024-04-26T21:19:45.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:45 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3878316915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:19:45.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:45 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3878316915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:19:45.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:45 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3878316915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:19:46.373 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:19:46.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:46.373 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.479+0000 7faf1812b640 1 -- 172.21.15.129:0/303661737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 msgr2=0x7faf10101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.479+0000 7faf1812b640 1 --2- 172.21.15.129:0/303661737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 0x7faf10101c90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009900 tx=0x7faf0c02a410 comp rx=0 tx=0).stop 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.479+0000 7faf1812b640 1 -- 172.21.15.129:0/303661737 shutdown_connections 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.479+0000 7faf1812b640 1 --2- 172.21.15.129:0/303661737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf10103420 0x7faf10109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.479+0000 7faf1812b640 1 --2- 172.21.15.129:0/303661737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf10102a60 0x7faf10102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.479+0000 7faf1812b640 1 --2- 172.21.15.129:0/303661737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 0x7faf10101c90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.479+0000 7faf1812b640 1 -- 172.21.15.129:0/303661737 >> 172.21.15.129:0/303661737 conn(0x7faf100fcde0 msgr2=0x7faf100ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 -- 172.21.15.129:0/303661737 shutdown_connections 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 -- 172.21.15.129:0/303661737 wait complete. 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 Processor -- start 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 -- start start 2024-04-26T21:19:46.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 0x7faf1006a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf10102a60 0x7faf1019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf10103420 0x7faf1019ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010d370 con 0x7faf10101870 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010d1f0 con 0x7faf10102a60 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf1812b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010d4f0 con 0x7faf10103420 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf15ec7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 0x7faf1006a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf15ec7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 0x7faf1006a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57088/0 (socket says 172.21.15.129:57088) 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf15ec7640 1 -- 172.21.15.129:0/2341952280 learned_addr learned my addr 172.21.15.129:0/2341952280 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf156c6640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf10102a60 0x7faf1019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.480+0000 7faf166c8640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf10103420 0x7faf1019ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf15ec7640 1 -- 172.21.15.129:0/2341952280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf10103420 msgr2=0x7faf1019ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf10103420 0x7faf1019ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf15ec7640 1 -- 172.21.15.129:0/2341952280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf10102a60 msgr2=0x7faf1019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf10102a60 0x7faf1019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf15ec7640 1 -- 172.21.15.129:0/2341952280 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf101a08a0 con 0x7faf10101870 2024-04-26T21:19:46.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf166c8640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf10103420 0x7faf1019ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:46.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 0x7faf1006a630 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7faf0c006010 tx=0x7faf0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf06ffd640 1 -- 172.21.15.129:0/2341952280 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c03f030 con 0x7faf10101870 2024-04-26T21:19:46.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf06ffd640 1 -- 172.21.15.129:0/2341952280 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c0318f0 con 0x7faf10101870 2024-04-26T21:19:46.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf1812b640 1 -- 172.21.15.129:0/2341952280 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf101a0ad0 con 0x7faf10101870 2024-04-26T21:19:46.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf06ffd640 1 -- 172.21.15.129:0/2341952280 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c035070 con 0x7faf10101870 2024-04-26T21:19:46.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.481+0000 7faf1812b640 1 -- 172.21.15.129:0/2341952280 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf101a0fc0 con 0x7faf10101870 2024-04-26T21:19:46.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.483+0000 7faf06ffd640 1 -- 172.21.15.129:0/2341952280 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7faf0c042030 con 0x7faf10101870 2024-04-26T21:19:46.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.483+0000 7faf1ae49740 1 Processor -- start 2024-04-26T21:19:46.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.483+0000 7faf1ae49740 1 -- start start 2024-04-26T21:19:46.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f4cc0 0x556b2c6f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f56b0 0x556b2c6fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 0x556b2c6fccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf1ae49740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556b2c655b00 con 0x556b2c6f56b0 2024-04-26T21:19:46.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf1ae49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556b2c5f1920 con 0x556b2c6fa820 2024-04-26T21:19:46.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf1ae49740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556b2c651ac0 con 0x556b2c6f4cc0 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf15ec7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f56b0 0x556b2c6fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf15ec7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f56b0 0x556b2c6fa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57104/0 (socket says 172.21.15.129:57104) 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf156c6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 0x556b2c6fccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf15ec7640 1 -- 172.21.15.129:0/2264691839 learned_addr learned my addr 172.21.15.129:0/2264691839 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf166c8640 1 --2- 172.21.15.129:0/2264691839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f4cc0 0x556b2c6f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf06ffd640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faefc0a3c40 0x7faefc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf15ec7640 1 -- 172.21.15.129:0/2264691839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f4cc0 msgr2=0x556b2c6f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2264691839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f4cc0 0x556b2c6f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf15ec7640 1 -- 172.21.15.129:0/2264691839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 msgr2=0x556b2c6fccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2264691839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 0x556b2c6fccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf15ec7640 1 -- 172.21.15.129:0/2264691839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b2c42f7e0 con 0x556b2c6f56b0 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf156c6640 1 --2- 172.21.15.129:0/2264691839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 0x556b2c6fccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf166c8640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faefc0a3c40 0x7faefc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf06ffd640 1 -- 172.21.15.129:0/2341952280 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7faf0c0f0b30 con 0x7faf10101870 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.484+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2264691839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f56b0 0x556b2c6fa1b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0ee410 tx=0x7faf0c08b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce6bd09cd70185f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faefbfff640 1 -- 172.21.15.129:0/2264691839 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0dbe60 con 0x556b2c6f56b0 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf166c8640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faefc0a3c40 0x7faefc0a6100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x556b2c6f5f10 tx=0x7faf08004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faefbfff640 1 -- 172.21.15.129:0/2264691839 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c088850 con 0x556b2c6f56b0 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faefbfff640 1 -- 172.21.15.129:0/2264691839 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0bd3a0 con 0x556b2c6f56b0 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf1ae49740 1 -- 172.21.15.129:0/2264691839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f56b0 msgr2=0x556b2c6fa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2264691839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f56b0 0x556b2c6fa1b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0ee410 tx=0x7faf0c08b6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf1ae49740 1 -- 172.21.15.129:0/2264691839 shutdown_connections 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2264691839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 0x556b2c6fccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2264691839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f56b0 0x556b2c6fa1b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2264691839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f4cc0 0x556b2c6f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf1ae49740 1 -- 172.21.15.129:0/2264691839 >> 172.21.15.129:0/2264691839 conn(0x556b2c6ebf20 msgr2=0x556b2c6ec320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf1ae49740 1 -- 172.21.15.129:0/2264691839 shutdown_connections 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.485+0000 7faf1ae49740 1 -- 172.21.15.129:0/2264691839 wait complete. 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf1ae49740 1 -- start start 2024-04-26T21:19:46.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf1ae49740 1 Processor -- start 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c6fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f56b0 0x556b2c6ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6fa820 0x556b2c706330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf1ae49740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556b2c6fd590 con 0x556b2c6f4cc0 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf1ae49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556b2c5ef5d0 con 0x556b2c6f56b0 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf1ae49740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556b2c6fd710 con 0x556b2c6fa820 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf1ae49740 10 client.0 _ll_register_callbacks cb 0x556b2c5dd080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf15ec7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c6fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf15ec7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c6fedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57106/0 (socket says 172.21.15.129:57106) 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf15ec7640 1 -- 172.21.15.129:0/2890464637 learned_addr learned my addr 172.21.15.129:0/2890464637 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf156c6640 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f56b0 0x556b2c6ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.486+0000 7faf166c8640 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6fa820 0x556b2c706330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf15ec7640 1 -- 172.21.15.129:0/2890464637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6fa820 msgr2=0x556b2c706330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6fa820 0x556b2c706330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf15ec7640 1 -- 172.21.15.129:0/2890464637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f56b0 msgr2=0x556b2c6ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f56b0 0x556b2c6ff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf15ec7640 1 -- 172.21.15.129:0/2890464637 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b2c48ff70 con 0x556b2c6f4cc0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf156c6640 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f56b0 0x556b2c6ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c6fedb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0ee410 tx=0x7faf0c03b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ec1a8dd3da2363 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faef97fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faef97fa640 1 -- 172.21.15.129:0/2890464637 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c049070 con 0x556b2c6f4cc0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faef97fa640 1 -- 172.21.15.129:0/2890464637 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7faf0c0bad40 con 0x556b2c6f4cc0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf1ae49740 1 -- 172.21.15.129:0/2890464637 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0c088850 con 0x556b2c6f4cc0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faef97fa640 1 -- 172.21.15.129:0/2890464637 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7faf0c0dbe60 con 0x556b2c6f4cc0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf1ae49740 10 client.15585 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf1ae49740 1 -- 172.21.15.129:0/2890464637 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556b2c327170 con 0x556b2c6f4cc0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faef97fa640 1 -- 172.21.15.129:0/2890464637 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c088850 con 0x556b2c6f4cc0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faef97fa640 1 -- 172.21.15.129:0/2890464637 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7faf0c0b9310 con 0x556b2c6f4cc0 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf1ae49740 10 client.15585 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf1ae49740 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:46.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf1ae49740 20 client.15585 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:46.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556b2c70a5a0 0x556b2c70c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf1ae49740 4 client.15585 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:19:46.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.487+0000 7faf1ae49740 1 -- 172.21.15.129:0/2890464637 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b2c663710 con 0x556b2c70a5a0 2024-04-26T21:19:46.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.488+0000 7faf156c6640 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556b2c70a5a0 0x556b2c70c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.488+0000 7faef97fa640 10 client.15585 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:46.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.488+0000 7faf156c6640 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556b2c70a5a0 0x556b2c70c9e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8432462d7073e4cc server_cookie=f07202a4d852fb78 in_seq=0 out_seq=0 2024-04-26T21:19:46.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faef97fa640 1 -- 172.21.15.129:0/2890464637 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b2c663710 con 0x556b2c70a5a0 2024-04-26T21:19:46.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faef97fa640 10 client.15585 handle_command_reply: tid=0 2024-04-26T21:19:46.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 client.15585 shutdown 2024-04-26T21:19:46.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 -- 172.21.15.129:0/2890464637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 msgr2=0x556b2c6fedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c6fedb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0ee410 tx=0x7faf0c03b5d0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 -- 172.21.15.129:0/2890464637 shutdown_connections 2024-04-26T21:19:46.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556b2c70a5a0 0x556b2c70c9e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f56b0 0x556b2c6ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6fa820 0x556b2c706330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faef97fa640 0 client.15585 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:46.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2890464637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c6fedb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 -- 172.21.15.129:0/2890464637 >> 172.21.15.129:0/2890464637 conn(0x556b2c6ebf20 msgr2=0x556b2c6eb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 -- 172.21.15.129:0/2890464637 shutdown_connections 2024-04-26T21:19:46.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 1 -- 172.21.15.129:0/2890464637 wait complete. 2024-04-26T21:19:46.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.492+0000 7faf1ae49740 20 client.15585 trim_cache size 0 max 16384 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf1ae49740 1 Processor -- start 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf1ae49740 1 -- start start 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f4cc0 0x556b2c6ff250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 0x556b2c7062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6fa820 0x556b2c6eaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf1ae49740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf101a5c70 con 0x556b2c6fa820 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf1ae49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf101a84f0 con 0x556b2c6f4cc0 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf1ae49740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faf101a62e0 con 0x556b2c6f56b0 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf166c8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6fa820 0x556b2c6eaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf166c8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6fa820 0x556b2c6eaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57110/0 (socket says 172.21.15.129:57110) 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf166c8640 1 -- 172.21.15.129:0/2291190673 learned_addr learned my addr 172.21.15.129:0/2291190673 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf15ec7640 1 --2- 172.21.15.129:0/2291190673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f4cc0 0x556b2c6ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf156c6640 1 --2- 172.21.15.129:0/2291190673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 0x556b2c7062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf166c8640 1 -- 172.21.15.129:0/2291190673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 msgr2=0x556b2c7062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf166c8640 1 --2- 172.21.15.129:0/2291190673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 0x556b2c7062c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf166c8640 1 -- 172.21.15.129:0/2291190673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f4cc0 msgr2=0x556b2c6ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf166c8640 1 --2- 172.21.15.129:0/2291190673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f4cc0 0x556b2c6ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.510+0000 7faf166c8640 1 -- 172.21.15.129:0/2291190673 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b2c6f6600 con 0x556b2c6fa820 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf166c8640 1 --2- 172.21.15.129:0/2291190673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6fa820 0x556b2c6eaf60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faf080105d0 tx=0x7faf08018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8e0ef8c8bdc47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faefb7fe640 1 -- 172.21.15.129:0/2291190673 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0802c070 con 0x556b2c6fa820 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faefb7fe640 1 -- 172.21.15.129:0/2291190673 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf08010d40 con 0x556b2c6fa820 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faefb7fe640 1 -- 172.21.15.129:0/2291190673 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080116c0 con 0x556b2c6fa820 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf1ae49740 1 -- 172.21.15.129:0/2291190673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6fa820 msgr2=0x556b2c6eaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2291190673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6fa820 0x556b2c6eaf60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faf080105d0 tx=0x7faf08018000 comp rx=0 tx=0).stop 2024-04-26T21:19:46.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf1ae49740 1 -- 172.21.15.129:0/2291190673 shutdown_connections 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2291190673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6fa820 0x556b2c6eaf60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2291190673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 0x556b2c7062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf1ae49740 1 --2- 172.21.15.129:0/2291190673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6f4cc0 0x556b2c6ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf1ae49740 1 -- 172.21.15.129:0/2291190673 >> 172.21.15.129:0/2291190673 conn(0x556b2c6ebf20 msgr2=0x556b2c6ea850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf1ae49740 1 -- 172.21.15.129:0/2291190673 shutdown_connections 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.511+0000 7faf1ae49740 1 -- 172.21.15.129:0/2291190673 wait complete. 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf1ae49740 1 -- start start 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf1ae49740 1 Processor -- start 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c704020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 0x556b2c704560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf1ae49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 0x556b2c705570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf1ae49740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf101a69b0 con 0x556b2c6f4cc0 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf1ae49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf101a7e20 con 0x556b2c6fa820 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf1ae49740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faf101a7080 con 0x556b2c6f56b0 2024-04-26T21:19:46.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf1ae49740 10 client.0 _ll_register_callbacks cb 0x7faf0c0b8e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf15ec7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c704020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf15ec7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c704020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57114/0 (socket says 172.21.15.129:57114) 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf15ec7640 1 -- 172.21.15.129:0/796204203 learned_addr learned my addr 172.21.15.129:0/796204203 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf166c8640 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 0x556b2c705570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf156c6640 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 0x556b2c704560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf15ec7640 1 -- 172.21.15.129:0/796204203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 msgr2=0x556b2c704560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf15ec7640 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 0x556b2c704560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf15ec7640 1 -- 172.21.15.129:0/796204203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 msgr2=0x556b2c705570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf15ec7640 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 0x556b2c705570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.512+0000 7faf15ec7640 1 -- 172.21.15.129:0/796204203 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b2c705b60 con 0x556b2c6f4cc0 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf15ec7640 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c704020 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faf0c087e90 tx=0x7faf0c0bafa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c609740a1525a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faef97fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faef97fa640 1 -- 172.21.15.129:0/796204203 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c049050 con 0x556b2c6f4cc0 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faef97fa640 1 -- 172.21.15.129:0/796204203 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7faf0c0ee720 con 0x556b2c6f4cc0 2024-04-26T21:19:46.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf1ae49740 1 -- 172.21.15.129:0/796204203 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf08010d40 con 0x556b2c6f4cc0 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faef97fa640 1 -- 172.21.15.129:0/796204203 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7faf0c0bc600 con 0x556b2c6f4cc0 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf1ae49740 10 client.15591 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf1ae49740 1 -- 172.21.15.129:0/796204203 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556b2c705b60 con 0x556b2c6f4cc0 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faef97fa640 1 -- 172.21.15.129:0/796204203 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf08010d40 con 0x556b2c6f4cc0 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faef97fa640 1 -- 172.21.15.129:0/796204203 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7faf0c08b0f0 con 0x556b2c6f4cc0 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf1ae49740 10 client.15591 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf1ae49740 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf1ae49740 20 client.15591 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf1ae49740 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556b2c7716f0 0x556b2c773b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf1ae49740 4 client.15591 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf1ae49740 1 -- 172.21.15.129:0/796204203 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556b2c65d580 con 0x556b2c7716f0 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.513+0000 7faf156c6640 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556b2c7716f0 0x556b2c773b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.514+0000 7faef97fa640 10 client.15591 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:46.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.514+0000 7faf156c6640 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556b2c7716f0 0x556b2c773b10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b4d3cfaead0c1 server_cookie=f7feda311955f5ae in_seq=0 out_seq=0 2024-04-26T21:19:46.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.551+0000 7faef97fa640 1 -- 172.21.15.129:0/796204203 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556b2c65d580 con 0x556b2c7716f0 2024-04-26T21:19:46.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.551+0000 7faef97fa640 10 client.15591 handle_command_reply: tid=0 2024-04-26T21:19:46.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.551+0000 7faf1ae49740 1 client.15591 shutdown 2024-04-26T21:19:46.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 -- 172.21.15.129:0/796204203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 msgr2=0x556b2c704020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c704020 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faf0c087e90 tx=0x7faf0c0bafa0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 -- 172.21.15.129:0/796204203 shutdown_connections 2024-04-26T21:19:46.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556b2c7716f0 0x556b2c773b10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b2c6fa820 0x556b2c705570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b2c6f56b0 0x556b2c704560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 --2- 172.21.15.129:0/796204203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556b2c6f4cc0 0x556b2c704020 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faef97fa640 0 client.15591 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:46.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 -- 172.21.15.129:0/796204203 >> 172.21.15.129:0/796204203 conn(0x556b2c6ebf20 msgr2=0x556b2c6ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 -- 172.21.15.129:0/796204203 shutdown_connections 2024-04-26T21:19:46.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 1 -- 172.21.15.129:0/796204203 wait complete. 2024-04-26T21:19:46.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1ae49740 20 client.15591 trim_cache size 0 max 16384 2024-04-26T21:19:46.555 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:19:46.555 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:19:46.555 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "850558e2-45f0-4562-84bf-24935311c445", 2024-04-26T21:19:46.555 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:19:46.555 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:19:46.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 -- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faefc0a3c40 msgr2=0x7faefc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faefc0a3c40 0x7faefc0a6100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x556b2c6f5f10 tx=0x7faf08004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:46.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 -- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 msgr2=0x7faf1006a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 0x7faf1006a630 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7faf0c006010 tx=0x7faf0c0042f0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 -- 172.21.15.129:0/2341952280 shutdown_connections 2024-04-26T21:19:46.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7faefc0a3c40 0x7faefc0a6100 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf10102a60 0x7faf1019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf10103420 0x7faf1019ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 --2- 172.21.15.129:0/2341952280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf10101870 0x7faf1006a630 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 -- 172.21.15.129:0/2341952280 >> 172.21.15.129:0/2341952280 conn(0x7faf100fcde0 msgr2=0x7faf10105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.552+0000 7faf1812b640 1 -- 172.21.15.129:0/2341952280 shutdown_connections 2024-04-26T21:19:46.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.553+0000 7faf1812b640 1 -- 172.21.15.129:0/2341952280 wait complete. 2024-04-26T21:19:46.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:19:46.565 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.668+0000 7f5efb33f640 1 -- 172.21.15.129:0/3841945276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 msgr2=0x7f5ef4101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.668+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3841945276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 0x7f5ef4101c90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009ca0 tx=0x7f5ef002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.668+0000 7f5efb33f640 1 -- 172.21.15.129:0/3841945276 shutdown_connections 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.668+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3841945276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ef4102a60 0x7f5ef4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.668+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3841945276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ef4103420 0x7f5ef4109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.668+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3841945276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 0x7f5ef4101c90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.668+0000 7f5efb33f640 1 -- 172.21.15.129:0/3841945276 >> 172.21.15.129:0/3841945276 conn(0x7f5ef40fcde0 msgr2=0x7f5ef40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 -- 172.21.15.129:0/3841945276 shutdown_connections 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 -- 172.21.15.129:0/3841945276 wait complete. 2024-04-26T21:19:46.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 Processor -- start 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 -- start start 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 0x7f5ef4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ef4102a60 0x7f5ef4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ef4103420 0x7f5ef41a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef410e4c0 con 0x7f5ef4101870 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef410e340 con 0x7f5ef4102a60 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5efb33f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef410e640 con 0x7f5ef4103420 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5ef90db640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 0x7f5ef4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5ef90db640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 0x7f5ef4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57142/0 (socket says 172.21.15.129:57142) 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5ef90db640 1 -- 172.21.15.129:0/3996144719 learned_addr learned my addr 172.21.15.129:0/3996144719 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5ef88da640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ef4102a60 0x7f5ef4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.669+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ef4103420 0x7f5ef41a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5ef90db640 1 -- 172.21.15.129:0/3996144719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ef4103420 msgr2=0x7f5ef41a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5ef90db640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ef4103420 0x7f5ef41a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5ef90db640 1 -- 172.21.15.129:0/3996144719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ef4102a60 msgr2=0x7f5ef4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5ef90db640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ef4102a60 0x7f5ef4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5ef90db640 1 -- 172.21.15.129:0/3996144719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef41a0860 con 0x7f5ef4101870 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5ef90db640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 0x7f5ef4198c50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009430 tx=0x7f5ef0031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5eea7fc640 1 -- 172.21.15.129:0/3996144719 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0003cb0 con 0x7f5ef4101870 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5eea7fc640 1 -- 172.21.15.129:0/3996144719 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef0003e50 con 0x7f5ef4101870 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5efb33f640 1 -- 172.21.15.129:0/3996144719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef41a0af0 con 0x7f5ef4101870 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5eea7fc640 1 -- 172.21.15.129:0/3996144719 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef002fc60 con 0x7f5ef4101870 2024-04-26T21:19:46.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.670+0000 7f5efb33f640 1 -- 172.21.15.129:0/3996144719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef41a1040 con 0x7f5ef4101870 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.672+0000 7f5eea7fc640 1 -- 172.21.15.129:0/3996144719 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5ef0004040 con 0x7f5ef4101870 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5efe058740 1 Processor -- start 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5efe058740 1 -- start start 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5eea7fc640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5ee00a3c40 0x7f5ee00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5efe058740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc21300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5efe058740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 0x55b59bc22530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5efe058740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc29300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5ee00a3c40 0x7f5ee00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5efe058740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b59bb85d80 con 0x55b59bc20ee0 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5efe058740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b59bb3f8b0 con 0x55b59bc22a70 2024-04-26T21:19:46.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5efe058740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b59bb81d40 con 0x55b59bc220b0 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5eea7fc640 1 -- 172.21.15.129:0/3996144719 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5ef00f1a50 con 0x7f5ef4101870 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef90db640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc21300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef90db640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc21300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57156/0 (socket says 172.21.15.129:57156) 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef90db640 1 -- 172.21.15.129:0/2125256445 learned_addr learned my addr 172.21.15.129:0/2125256445 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/2125256445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc29300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef88da640 1 --2- 172.21.15.129:0/2125256445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 0x55b59bc22530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef90db640 1 -- 172.21.15.129:0/2125256445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 msgr2=0x55b59bc22530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef90db640 1 --2- 172.21.15.129:0/2125256445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 0x55b59bc22530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef90db640 1 -- 172.21.15.129:0/2125256445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 msgr2=0x55b59bc29300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef90db640 1 --2- 172.21.15.129:0/2125256445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc29300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef90db640 1 -- 172.21.15.129:0/2125256445 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59bb10a40 con 0x55b59bc20ee0 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.673+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5ee00a3c40 0x7f5ee00a6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5eec004650 tx=0x7f5eec00bdc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.674+0000 7f5ef90db640 1 --2- 172.21.15.129:0/2125256445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc21300 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00ba610 tx=0x7f5ef00bdb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a8b639a204d47c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.674+0000 7f5edf7fe640 1 -- 172.21.15.129:0/2125256445 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00be3d0 con 0x55b59bc20ee0 2024-04-26T21:19:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.674+0000 7f5edf7fe640 1 -- 172.21.15.129:0/2125256445 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef0036040 con 0x55b59bc20ee0 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.674+0000 7f5edf7fe640 1 -- 172.21.15.129:0/2125256445 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00c2080 con 0x55b59bc20ee0 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.674+0000 7f5efe058740 1 -- 172.21.15.129:0/2125256445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 msgr2=0x55b59bc21300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.674+0000 7f5efe058740 1 --2- 172.21.15.129:0/2125256445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc21300 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00ba610 tx=0x7f5ef00bdb60 comp rx=0 tx=0).stop 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.674+0000 7f5efe058740 1 -- 172.21.15.129:0/2125256445 shutdown_connections 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.674+0000 7f5efe058740 1 --2- 172.21.15.129:0/2125256445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc29300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.674+0000 7f5efe058740 1 --2- 172.21.15.129:0/2125256445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 0x55b59bc22530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 --2- 172.21.15.129:0/2125256445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc21300 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 -- 172.21.15.129:0/2125256445 >> 172.21.15.129:0/2125256445 conn(0x55b59bc1c3d0 msgr2=0x55b59bc1c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 -- 172.21.15.129:0/2125256445 shutdown_connections 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 -- 172.21.15.129:0/2125256445 wait complete. 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 -- start start 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 Processor -- start 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc2f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 0x55b59bc2f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b59bb85d80 con 0x55b59bc20ee0 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b59bb0cd10 con 0x55b59bc22a70 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b59bc2dc50 con 0x55b59bc220b0 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5efe058740 10 client.0 _ll_register_callbacks cb 0x55b59bb39210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5ef90db640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc2f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5ef90db640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc2f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57170/0 (socket says 172.21.15.129:57170) 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5ef90db640 1 -- 172.21.15.129:0/2436573916 learned_addr learned my addr 172.21.15.129:0/2436573916 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.675+0000 7f5ef88da640 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 0x55b59bc2f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5ef90db640 1 -- 172.21.15.129:0/2436573916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 msgr2=0x55b59bc2f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5ef90db640 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 0x55b59bc2f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5ef90db640 1 -- 172.21.15.129:0/2436573916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 msgr2=0x55b59bc368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5ef90db640 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5ef90db640 1 -- 172.21.15.129:0/2436573916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59b9617d0 con 0x55b59bc20ee0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5ef90db640 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc2f320 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00ef580 tx=0x7f5ef00492a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965303e16efde96c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5edcff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5edcff9640 1 -- 172.21.15.129:0/2436573916 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00d6070 con 0x55b59bc20ee0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5edcff9640 1 -- 172.21.15.129:0/2436573916 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5ef00ef5b0 con 0x55b59bc20ee0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5efe058740 1 -- 172.21.15.129:0/2436573916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef0036040 con 0x55b59bc20ee0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5edcff9640 1 -- 172.21.15.129:0/2436573916 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5ef00c6940 con 0x55b59bc20ee0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5efe058740 10 client.15603 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5efe058740 1 -- 172.21.15.129:0/2436573916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b59b8973f0 con 0x55b59bc20ee0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5edcff9640 1 -- 172.21.15.129:0/2436573916 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef0036040 con 0x55b59bc20ee0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5edcff9640 1 -- 172.21.15.129:0/2436573916 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f5ef0004330 con 0x55b59bc20ee0 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5efe058740 10 client.15603 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5efe058740 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5efe058740 20 client.15603 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5efe058740 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b59bc3ade0 0x55b59bc3d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5efe058740 4 client.15603 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:19:46.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.676+0000 7f5efe058740 1 -- 172.21.15.129:0/2436573916 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b59bb93990 con 0x55b59bc3ade0 2024-04-26T21:19:46.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.677+0000 7f5ef88da640 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b59bc3ade0 0x55b59bc3d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.677+0000 7f5ef88da640 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b59bc3ade0 0x55b59bc3d210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c71d211776712fb server_cookie=d9fc171502b1db28 in_seq=0 out_seq=0 2024-04-26T21:19:46.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.677+0000 7f5edcff9640 10 client.15603 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:46.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5edcff9640 1 -- 172.21.15.129:0/2436573916 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b59bb93990 con 0x55b59bc3ade0 2024-04-26T21:19:46.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5edcff9640 10 client.15603 handle_command_reply: tid=0 2024-04-26T21:19:46.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5efe058740 1 client.15603 shutdown 2024-04-26T21:19:46.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5efe058740 1 -- 172.21.15.129:0/2436573916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 msgr2=0x55b59bc2f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5efe058740 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc2f320 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00ef580 tx=0x7f5ef00492a0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5efe058740 1 -- 172.21.15.129:0/2436573916 shutdown_connections 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5efe058740 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b59bc3ade0 0x55b59bc3d210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5efe058740 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5edcff9640 0 client.15603 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5efe058740 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc220b0 0x55b59bc2f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5efe058740 1 --2- 172.21.15.129:0/2436573916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc20ee0 0x55b59bc2f320 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.681+0000 7f5efe058740 1 -- 172.21.15.129:0/2436573916 >> 172.21.15.129:0/2436573916 conn(0x55b59bc1c3d0 msgr2=0x55b59bc1b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.682+0000 7f5efe058740 1 -- 172.21.15.129:0/2436573916 shutdown_connections 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.682+0000 7f5efe058740 1 -- 172.21.15.129:0/2436573916 wait complete. 2024-04-26T21:19:46.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.682+0000 7f5efe058740 20 client.15603 trim_cache size 0 max 16384 2024-04-26T21:19:46.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5efe058740 1 Processor -- start 2024-04-26T21:19:46.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5efe058740 1 -- start start 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5efe058740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 0x55b59bc1bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5efe058740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc220b0 0x55b59bc36830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5efe058740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc2f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5efe058740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef41a5b40 con 0x55b59bc220b0 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5efe058740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef41a83c0 con 0x55b59bc22a70 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5efe058740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef41a61b0 con 0x55b59bc20ee0 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5ef88da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc220b0 0x55b59bc36830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5ef98dc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc2f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5ef88da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc220b0 0x55b59bc36830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57172/0 (socket says 172.21.15.129:57172) 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5ef88da640 1 -- 172.21.15.129:0/730108333 learned_addr learned my addr 172.21.15.129:0/730108333 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.699+0000 7f5ef90db640 1 --2- 172.21.15.129:0/730108333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 0x55b59bc1bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5ef88da640 1 -- 172.21.15.129:0/730108333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 msgr2=0x55b59bc1bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5ef88da640 1 --2- 172.21.15.129:0/730108333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 0x55b59bc1bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5ef88da640 1 -- 172.21.15.129:0/730108333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 msgr2=0x55b59bc2f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5ef88da640 1 --2- 172.21.15.129:0/730108333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc2f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5ef88da640 1 -- 172.21.15.129:0/730108333 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59b8570f0 con 0x55b59bc220b0 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/730108333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5ef88da640 1 --2- 172.21.15.129:0/730108333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc220b0 0x55b59bc36830 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005d00 tx=0x7f5ee4005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888393844a8787ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5edeffd640 1 -- 172.21.15.129:0/730108333 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4004020 con 0x55b59bc220b0 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5edeffd640 1 -- 172.21.15.129:0/730108333 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee401d8e0 con 0x55b59bc220b0 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5edeffd640 1 -- 172.21.15.129:0/730108333 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4005520 con 0x55b59bc220b0 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5efe058740 1 -- 172.21.15.129:0/730108333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc220b0 msgr2=0x55b59bc36830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.700+0000 7f5efe058740 1 --2- 172.21.15.129:0/730108333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc220b0 0x55b59bc36830 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005d00 tx=0x7f5ee4005900 comp rx=0 tx=0).stop 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 -- 172.21.15.129:0/730108333 shutdown_connections 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 --2- 172.21.15.129:0/730108333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc22a70 0x55b59bc2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 --2- 172.21.15.129:0/730108333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc220b0 0x55b59bc36830 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 --2- 172.21.15.129:0/730108333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 0x55b59bc1bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 -- 172.21.15.129:0/730108333 >> 172.21.15.129:0/730108333 conn(0x55b59bc1c3d0 msgr2=0x55b59bc1f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 -- 172.21.15.129:0/730108333 shutdown_connections 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 -- 172.21.15.129:0/730108333 wait complete. 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 -- start start 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 Processor -- start 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 0x55b59bc33ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc220b0 0x55b59bc341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc22a70 0x55b59bc351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef41a6880 con 0x55b59bc22a70 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef41a7cf0 con 0x55b59bc220b0 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef41a6f50 con 0x55b59bc20ee0 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5efe058740 10 client.0 _ll_register_callbacks cb 0x7f5ef00b9f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5ef98dc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc22a70 0x55b59bc351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5ef98dc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc22a70 0x55b59bc351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57174/0 (socket says 172.21.15.129:57174) 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5ef98dc640 1 -- 172.21.15.129:0/1369429692 learned_addr learned my addr 172.21.15.129:0/1369429692 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5ef90db640 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 0x55b59bc33ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.701+0000 7f5ef88da640 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc220b0 0x55b59bc341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5ef98dc640 1 -- 172.21.15.129:0/1369429692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 msgr2=0x55b59bc33ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 0x55b59bc33ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5ef98dc640 1 -- 172.21.15.129:0/1369429692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc220b0 msgr2=0x55b59bc341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc220b0 0x55b59bc341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5ef98dc640 1 -- 172.21.15.129:0/1369429692 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59bb464b0 con 0x55b59bc22a70 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5ef90db640 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 0x55b59bc33ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5ef98dc640 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc22a70 0x55b59bc351f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5eec011940 tx=0x7f5eec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb618fe0f6dd639 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5edcff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5edcff9640 1 -- 172.21.15.129:0/1369429692 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec022b90 con 0x55b59bc22a70 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5edcff9640 1 -- 172.21.15.129:0/1369429692 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5eec022d30 con 0x55b59bc22a70 2024-04-26T21:19:46.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5efe058740 1 -- 172.21.15.129:0/1369429692 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee401d8e0 con 0x55b59bc22a70 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5edcff9640 1 -- 172.21.15.129:0/1369429692 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5eec02b440 con 0x55b59bc22a70 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5efe058740 10 client.15609 fetch_fsmap learned FSMap version 19 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5efe058740 1 -- 172.21.15.129:0/1369429692 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b59bc369c0 con 0x55b59bc22a70 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5edcff9640 1 -- 172.21.15.129:0/1369429692 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee401d8e0 con 0x55b59bc22a70 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.702+0000 7f5edcff9640 1 -- 172.21.15.129:0/1369429692 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f5eec034050 con 0x55b59bc22a70 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.703+0000 7f5efe058740 10 client.15609 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.703+0000 7f5efe058740 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.703+0000 7f5efe058740 20 client.15609 populate_metadata read hostname 'smithi129' 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.703+0000 7f5efe058740 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b59bca19f0 0x55b59bca3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.703+0000 7f5efe058740 4 client.15609 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.703+0000 7f5efe058740 1 -- 172.21.15.129:0/1369429692 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b59bb8d800 con 0x55b59bca19f0 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.703+0000 7f5ef90db640 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b59bca19f0 0x55b59bca3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.703+0000 7f5edcff9640 10 client.15609 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:46.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.703+0000 7f5ef90db640 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b59bca19f0 0x55b59bca3e10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813d8ce6dae6fe3d server_cookie=8e80eb1fa9977352 in_seq=0 out_seq=0 2024-04-26T21:19:46.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5edcff9640 1 -- 172.21.15.129:0/1369429692 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55b59bb8d800 con 0x55b59bca19f0 2024-04-26T21:19:46.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5edcff9640 10 client.15609 handle_command_reply: tid=0 2024-04-26T21:19:46.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 client.15609 shutdown 2024-04-26T21:19:46.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 -- 172.21.15.129:0/1369429692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc22a70 msgr2=0x55b59bc351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc22a70 0x55b59bc351f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5eec011940 tx=0x7f5eec004730 comp rx=0 tx=0).stop 2024-04-26T21:19:46.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 -- 172.21.15.129:0/1369429692 shutdown_connections 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b59bca19f0 0x55b59bca3e10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b59bc220b0 0x55b59bc341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b59bc22a70 0x55b59bc351f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 --2- 172.21.15.129:0/1369429692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b59bc20ee0 0x55b59bc33ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5edcff9640 0 client.15609 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 -- 172.21.15.129:0/1369429692 >> 172.21.15.129:0/1369429692 conn(0x55b59bc1c3d0 msgr2=0x55b59bc2f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 -- 172.21.15.129:0/1369429692 shutdown_connections 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 1 -- 172.21.15.129:0/1369429692 wait complete. 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.711+0000 7f5efe058740 20 client.15609 trim_cache size 0 max 16384 2024-04-26T21:19:46.713 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:19:46.714 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (30 inodes in the stack)", 2024-04-26T21:19:46.714 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:19:46.714 INFO:teuthology.orchestra.run.smithi129.stdout: "850558e2-45f0-4562-84bf-24935311c445": { 2024-04-26T21:19:46.714 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:19:46.714 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "850558e2-45f0-4562-84bf-24935311c445", 2024-04-26T21:19:46.714 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:19:46.714 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:19:46.714 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:19:46.714 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:19:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 -- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5ee00a3c40 msgr2=0x7f5ee00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5ee00a3c40 0x7f5ee00a6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5eec004650 tx=0x7f5eec00bdc0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 -- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 msgr2=0x7f5ef4198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 0x7f5ef4198c50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009430 tx=0x7f5ef0031840 comp rx=0 tx=0).stop 2024-04-26T21:19:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 -- 172.21.15.129:0/3996144719 shutdown_connections 2024-04-26T21:19:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5ee00a3c40 0x7f5ee00a6100 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ef4102a60 0x7f5ef4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ef4103420 0x7f5ef41a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 --2- 172.21.15.129:0/3996144719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ef4101870 0x7f5ef4198c50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 -- 172.21.15.129:0/3996144719 >> 172.21.15.129:0/3996144719 conn(0x7f5ef40fcde0 msgr2=0x7f5ef4105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 -- 172.21.15.129:0/3996144719 shutdown_connections 2024-04-26T21:19:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:19:46.712+0000 7f5efb33f640 1 -- 172.21.15.129:0/3996144719 wait complete. 2024-04-26T21:19:46.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:46 smithi129 ceph-mon[29237]: pgmap v403: 97 pgs: 97 active+clean; 5.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 515 KiB/s wr, 51 op/s 2024-04-26T21:19:46.724 INFO:tasks.cephfs.filesystem:scrub status for tag:850558e2-45f0-4562-84bf-24935311c445 - {'path': '/', 'tag': '850558e2-45f0-4562-84bf-24935311c445', 'options': 'recursive,force'} 2024-04-26T21:19:46.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:46 smithi171 ceph-mon[33478]: pgmap v403: 97 pgs: 97 active+clean; 5.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 515 KiB/s wr, 51 op/s 2024-04-26T21:19:46.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:46 smithi150 ceph-mon[32290]: pgmap v403: 97 pgs: 97 active+clean; 5.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 515 KiB/s wr, 51 op/s 2024-04-26T21:19:47.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:47 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:19:47.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:47 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:47.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:47 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:19:47.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:47 smithi129 ceph-mon[29237]: pgmap v404: 97 pgs: 97 active+clean; 9.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 890 KiB/s wr, 59 op/s 2024-04-26T21:19:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:47 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:19:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:47 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:47 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:19:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:47 smithi171 ceph-mon[33478]: pgmap v404: 97 pgs: 97 active+clean; 9.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 890 KiB/s wr, 59 op/s 2024-04-26T21:19:47.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:47 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:19:47.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:47 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:47 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:19:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:47 smithi150 ceph-mon[32290]: pgmap v404: 97 pgs: 97 active+clean; 9.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 890 KiB/s wr, 59 op/s 2024-04-26T21:19:50.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:50 smithi129 ceph-mon[29237]: pgmap v405: 97 pgs: 97 active+clean; 29 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-04-26T21:19:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:50 smithi171 ceph-mon[33478]: pgmap v405: 97 pgs: 97 active+clean; 29 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-04-26T21:19:50.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:50 smithi150 ceph-mon[32290]: pgmap v405: 97 pgs: 97 active+clean; 29 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-04-26T21:19:51.767 INFO:tasks.workunit.client.0.smithi129.stdout: 12 213788 687 139334 719 95430 1126 2024-04-26T21:19:52.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:52 smithi129 ceph-mon[29237]: pgmap v406: 97 pgs: 97 active+clean; 39 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-04-26T21:19:52.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:52 smithi171 ceph-mon[33478]: pgmap v406: 97 pgs: 97 active+clean; 39 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-04-26T21:19:52.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:52 smithi150 ceph-mon[32290]: pgmap v406: 97 pgs: 97 active+clean; 39 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-04-26T21:19:54.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:54 smithi129 ceph-mon[29237]: pgmap v407: 97 pgs: 97 active+clean; 49 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 225 op/s 2024-04-26T21:19:54.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:54 smithi171 ceph-mon[33478]: pgmap v407: 97 pgs: 97 active+clean; 49 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 225 op/s 2024-04-26T21:19:54.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:54 smithi150 ceph-mon[32290]: pgmap v407: 97 pgs: 97 active+clean; 49 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 225 op/s 2024-04-26T21:19:55.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:55 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:55.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:55 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:55.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:55 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:19:56.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:56 smithi129 ceph-mon[29237]: pgmap v408: 97 pgs: 97 active+clean; 49 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-04-26T21:19:56.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:56 smithi171 ceph-mon[33478]: pgmap v408: 97 pgs: 97 active+clean; 49 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-04-26T21:19:56.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:56 smithi150 ceph-mon[32290]: pgmap v408: 97 pgs: 97 active+clean; 49 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-04-26T21:19:57.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:57 smithi129 ceph-mon[29237]: pgmap v409: 97 pgs: 97 active+clean; 52 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 212 op/s 2024-04-26T21:19:57.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:57 smithi171 ceph-mon[33478]: pgmap v409: 97 pgs: 97 active+clean; 52 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 212 op/s 2024-04-26T21:19:57.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:57 smithi150 ceph-mon[32290]: pgmap v409: 97 pgs: 97 active+clean; 52 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 212 op/s 2024-04-26T21:19:59.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:19:59 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:19:59.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:19:59 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:19:59.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:19:59 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:20:00.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:00 smithi150 ceph-mon[32290]: pgmap v410: 97 pgs: 97 active+clean; 51 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 219 op/s 2024-04-26T21:20:00.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:00 smithi150 ceph-mon[32290]: overall HEALTH_OK 2024-04-26T21:20:00.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:00 smithi129 ceph-mon[29237]: pgmap v410: 97 pgs: 97 active+clean; 51 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 219 op/s 2024-04-26T21:20:00.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:00 smithi129 ceph-mon[29237]: overall HEALTH_OK 2024-04-26T21:20:00.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:00 smithi171 ceph-mon[33478]: pgmap v410: 97 pgs: 97 active+clean; 51 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 219 op/s 2024-04-26T21:20:00.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:00 smithi171 ceph-mon[33478]: overall HEALTH_OK 2024-04-26T21:20:01.767 INFO:tasks.workunit.client.0.smithi129.stdout: 21 615801 386 479212 520 296308 1639 2024-04-26T21:20:01.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:01 smithi171 ceph-mon[33478]: pgmap v411: 97 pgs: 97 active+clean; 59 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-04-26T21:20:02.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:01 smithi150 ceph-mon[32290]: pgmap v411: 97 pgs: 97 active+clean; 59 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-04-26T21:20:02.219 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:01 smithi129 ceph-mon[29237]: pgmap v411: 97 pgs: 97 active+clean; 59 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-04-26T21:20:04.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:04 smithi129 ceph-mon[29237]: pgmap v412: 97 pgs: 97 active+clean; 63 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 94 op/s 2024-04-26T21:20:04.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:04 smithi171 ceph-mon[33478]: pgmap v412: 97 pgs: 97 active+clean; 63 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 94 op/s 2024-04-26T21:20:04.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:04 smithi150 ceph-mon[32290]: pgmap v412: 97 pgs: 97 active+clean; 63 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 94 op/s 2024-04-26T21:20:06.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:06 smithi129 ceph-mon[29237]: pgmap v413: 97 pgs: 97 active+clean; 63 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 53 op/s 2024-04-26T21:20:06.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:06 smithi171 ceph-mon[33478]: pgmap v413: 97 pgs: 97 active+clean; 63 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 53 op/s 2024-04-26T21:20:06.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:06 smithi150 ceph-mon[32290]: pgmap v413: 97 pgs: 97 active+clean; 63 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 53 op/s 2024-04-26T21:20:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:08 smithi150 ceph-mon[32290]: pgmap v414: 97 pgs: 97 active+clean; 67 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 56 op/s 2024-04-26T21:20:08.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:08 smithi129 ceph-mon[29237]: pgmap v414: 97 pgs: 97 active+clean; 67 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 56 op/s 2024-04-26T21:20:08.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:08 smithi171 ceph-mon[33478]: pgmap v414: 97 pgs: 97 active+clean; 67 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 56 op/s 2024-04-26T21:20:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:10 smithi129 ceph-mon[29237]: pgmap v415: 97 pgs: 97 active+clean; 93 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 173 op/s 2024-04-26T21:20:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:10 smithi171 ceph-mon[33478]: pgmap v415: 97 pgs: 97 active+clean; 93 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 173 op/s 2024-04-26T21:20:10.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:10 smithi150 ceph-mon[32290]: pgmap v415: 97 pgs: 97 active+clean; 93 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 173 op/s 2024-04-26T21:20:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:11 smithi129 ceph-mon[29237]: pgmap v416: 97 pgs: 97 active+clean; 126 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 308 op/s 2024-04-26T21:20:11.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:11 smithi171 ceph-mon[33478]: pgmap v416: 97 pgs: 97 active+clean; 126 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 308 op/s 2024-04-26T21:20:11.767 INFO:tasks.workunit.client.0.smithi129.stdout: 32 603173 725 465151 563 292381 448 2024-04-26T21:20:11.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:11 smithi150 ceph-mon[32290]: pgmap v416: 97 pgs: 97 active+clean; 126 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 308 op/s 2024-04-26T21:20:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:14 smithi129 ceph-mon[29237]: pgmap v417: 97 pgs: 97 active+clean; 162 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 516 op/s 2024-04-26T21:20:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:14 smithi171 ceph-mon[33478]: pgmap v417: 97 pgs: 97 active+clean; 162 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 516 op/s 2024-04-26T21:20:14.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:14 smithi150 ceph-mon[32290]: pgmap v417: 97 pgs: 97 active+clean; 162 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 516 op/s 2024-04-26T21:20:16.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:16 smithi129 ceph-mon[29237]: pgmap v418: 97 pgs: 97 active+clean; 158 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.0 MiB/s wr, 557 op/s 2024-04-26T21:20:16.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:20:16.725 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:20:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:16 smithi171 ceph-mon[33478]: pgmap v418: 97 pgs: 97 active+clean; 158 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.0 MiB/s wr, 557 op/s 2024-04-26T21:20:16.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 -- 172.21.15.129:0/3827563791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f98103420 msgr2=0x7f3f98109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/3827563791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f98103420 0x7f3f98109cb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009ca0 tx=0x7f3f9002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 -- 172.21.15.129:0/3827563791 shutdown_connections 2024-04-26T21:20:16.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/3827563791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f98103420 0x7f3f98109cb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/3827563791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f98102a60 0x7f3f98102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/3827563791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f98101870 0x7f3f98101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 -- 172.21.15.129:0/3827563791 >> 172.21.15.129:0/3827563791 conn(0x7f3f980fce20 msgr2=0x7f3f980ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 -- 172.21.15.129:0/3827563791 shutdown_connections 2024-04-26T21:20:16.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 -- 172.21.15.129:0/3827563791 wait complete. 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.837+0000 7f3f9f36a640 1 Processor -- start 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9f36a640 1 -- start start 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9f36a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f98101870 0x7f3f9806b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9f36a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f98102a60 0x7f3f9819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9f36a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f98103420 0x7f3f9819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9f36a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9810d3a0 con 0x7f3f98102a60 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9f36a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9810d220 con 0x7f3f98101870 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9f36a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9810d520 con 0x7f3f98103420 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d106640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f98101870 0x7f3f9806b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9c905640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f98102a60 0x7f3f9819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d106640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f98101870 0x7f3f9806b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60718/0 (socket says 172.21.15.129:60718) 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d106640 1 -- 172.21.15.129:0/400544339 learned_addr learned my addr 172.21.15.129:0/400544339 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d907640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f98103420 0x7f3f9819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d106640 1 -- 172.21.15.129:0/400544339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f98103420 msgr2=0x7f3f9819fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d106640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f98103420 0x7f3f9819fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d106640 1 -- 172.21.15.129:0/400544339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f98102a60 msgr2=0x7f3f9819d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d106640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f98102a60 0x7f3f9819d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d106640 1 -- 172.21.15.129:0/400544339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f981a07a0 con 0x7f3f98101870 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.838+0000 7f3f9d907640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f98103420 0x7f3f9819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.839+0000 7f3f9d106640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f98101870 0x7f3f9806b870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800e9c0 tx=0x7f3f8800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.839+0000 7f3f8e7fc640 1 -- 172.21.15.129:0/400544339 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800cd80 con 0x7f3f98101870 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.839+0000 7f3f8e7fc640 1 -- 172.21.15.129:0/400544339 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f880035f0 con 0x7f3f98101870 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.839+0000 7f3f8e7fc640 1 -- 172.21.15.129:0/400544339 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88010690 con 0x7f3f98101870 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.839+0000 7f3f9f36a640 1 -- 172.21.15.129:0/400544339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f981a0a90 con 0x7f3f98101870 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.839+0000 7f3f9f36a640 1 -- 172.21.15.129:0/400544339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f981a0f90 con 0x7f3f98101870 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.841+0000 7f3f8e7fc640 1 -- 172.21.15.129:0/400544339 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3f880108f0 con 0x7f3f98101870 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.842+0000 7f3f8e7fc640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3f840a3c40 0x7f3f840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.842+0000 7f3fa2081740 1 Processor -- start 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.842+0000 7f3fa2081740 1 -- start start 2024-04-26T21:20:16.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9c905640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3f840a3c40 0x7f3f840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f8e7fc640 1 -- 172.21.15.129:0/400544339 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3f880d0aa0 con 0x7f3f98101870 2024-04-26T21:20:16.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 0x55b2e88f3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e88f4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e88f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3fa2081740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2e8857d80 con 0x55b2e88f49d0 2024-04-26T21:20:16.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3fa2081740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b2e88118b0 con 0x55b2e88f4030 2024-04-26T21:20:16.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3fa2081740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b2e8853d40 con 0x55b2e88f2e10 2024-04-26T21:20:16.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9d907640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e88f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9d907640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e88f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51880/0 (socket says 172.21.15.129:51880) 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9d907640 1 -- 172.21.15.129:0/3753067506 learned_addr learned my addr 172.21.15.129:0/3753067506 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9c905640 1 --2- 172.21.15.129:0/3753067506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e88f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9d106640 1 --2- 172.21.15.129:0/3753067506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 0x55b2e88f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9c905640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3f840a3c40 0x7f3f840a6100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f9806b5e0 tx=0x7f3f9400ba60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9d907640 1 -- 172.21.15.129:0/3753067506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 msgr2=0x55b2e88f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9d907640 1 --2- 172.21.15.129:0/3753067506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 0x55b2e88f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9d907640 1 -- 172.21.15.129:0/3753067506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 msgr2=0x55b2e88f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9d907640 1 --2- 172.21.15.129:0/3753067506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e88f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.843+0000 7f3f9d907640 1 -- 172.21.15.129:0/3753067506 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e87e2a40 con 0x55b2e88f49d0 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.844+0000 7f3f9d907640 1 --2- 172.21.15.129:0/3753067506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e88f5270 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3f90035000 tx=0x7f3f900093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2210b062e3160a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.844+0000 7f3f837fe640 1 -- 172.21.15.129:0/3753067506 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90031650 con 0x55b2e88f49d0 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.844+0000 7f3f837fe640 1 -- 172.21.15.129:0/3753067506 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f90036040 con 0x55b2e88f49d0 2024-04-26T21:20:16.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.844+0000 7f3f837fe640 1 -- 172.21.15.129:0/3753067506 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f900045b0 con 0x55b2e88f49d0 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 -- 172.21.15.129:0/3753067506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 msgr2=0x55b2e88f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3753067506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e88f5270 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3f90035000 tx=0x7f3f900093e0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 -- 172.21.15.129:0/3753067506 shutdown_connections 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3753067506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e88f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3753067506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e88f5270 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3753067506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 0x55b2e88f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 -- 172.21.15.129:0/3753067506 >> 172.21.15.129:0/3753067506 conn(0x55b2e88ee3d0 msgr2=0x55b2e88ee7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 -- 172.21.15.129:0/3753067506 shutdown_connections 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 -- 172.21.15.129:0/3753067506 wait complete. 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 -- start start 2024-04-26T21:20:16.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.845+0000 7f3fa2081740 1 Processor -- start 2024-04-26T21:20:16.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f2e10 0x55b2e8905b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 0x55b2e8906050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e890d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3fa2081740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2e88f5990 con 0x55b2e88f49d0 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3fa2081740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b2e87ded10 con 0x55b2e88f2e10 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3fa2081740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b2e88f5b10 con 0x55b2e88f4030 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3fa2081740 10 client.0 _ll_register_callbacks cb 0x55b2e880b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d907640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e890d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d106640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f2e10 0x55b2e8905b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d907640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e890d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51882/0 (socket says 172.21.15.129:51882) 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d106640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f2e10 0x55b2e8905b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60732/0 (socket says 172.21.15.129:60732) 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d907640 1 -- 172.21.15.129:0/3772613316 learned_addr learned my addr 172.21.15.129:0/3772613316 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9c905640 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 0x55b2e8906050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d106640 1 -- 172.21.15.129:0/3772613316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 msgr2=0x55b2e8906050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d106640 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 0x55b2e8906050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d106640 1 -- 172.21.15.129:0/3772613316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 msgr2=0x55b2e890d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d106640 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e890d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d106640 1 -- 172.21.15.129:0/3772613316 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e86337d0 con 0x55b2e88f2e10 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f9d106640 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f2e10 0x55b2e8905b10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ce3e0 tx=0x7f3f880a58e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b9801cb4314b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.846+0000 7f3f80ff9640 1 -- 172.21.15.129:0/3772613316 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880bd070 con 0x55b2e88f2e10 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3fa2081740 1 -- 172.21.15.129:0/3772613316 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f90036040 con 0x55b2e88f2e10 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3f80ff9640 1 -- 172.21.15.129:0/3772613316 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3f880a5390 con 0x55b2e88f2e10 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3f80ff9640 1 -- 172.21.15.129:0/3772613316 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3f880bd210 con 0x55b2e88f2e10 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3fa2081740 10 client.25226 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3fa2081740 1 -- 172.21.15.129:0/3772613316 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b2e85693f0 con 0x55b2e88f2e10 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3f80ff9640 1 -- 172.21.15.129:0/3772613316 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f90036040 con 0x55b2e88f2e10 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3f80ff9640 1 -- 172.21.15.129:0/3772613316 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f3f880880a0 con 0x55b2e88f2e10 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3fa2081740 10 client.25226 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3fa2081740 10 client.25226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:16.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3fa2081740 20 client.25226 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:16.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2e8911520 0x55b2e89139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3fa2081740 4 client.25226 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:16.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3fa2081740 1 -- 172.21.15.129:0/3772613316 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2e8865990 con 0x55b2e8911520 2024-04-26T21:20:16.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.847+0000 7f3f9c905640 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2e8911520 0x55b2e89139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.848+0000 7f3f80ff9640 10 client.25226 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:16.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.848+0000 7f3f9c905640 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2e8911520 0x55b2e89139c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=722dcb627051ef56 server_cookie=caf501be8aba5d3d in_seq=0 out_seq=0 2024-04-26T21:20:16.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3f80ff9640 1 -- 172.21.15.129:0/3772613316 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2e8865990 con 0x55b2e8911520 2024-04-26T21:20:16.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3f80ff9640 10 client.25226 handle_command_reply: tid=0 2024-04-26T21:20:16.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 client.25226 shutdown 2024-04-26T21:20:16.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 -- 172.21.15.129:0/3772613316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f2e10 msgr2=0x55b2e8905b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f2e10 0x55b2e8905b10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ce3e0 tx=0x7f3f880a58e0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 -- 172.21.15.129:0/3772613316 shutdown_connections 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2e8911520 0x55b2e89139c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e890d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3f80ff9640 0 client.25226 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 0x55b2e8906050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 --2- 172.21.15.129:0/3772613316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f2e10 0x55b2e8905b10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 -- 172.21.15.129:0/3772613316 >> 172.21.15.129:0/3772613316 conn(0x55b2e88ee3d0 msgr2=0x55b2e88eccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.851+0000 7f3fa2081740 1 -- 172.21.15.129:0/3772613316 shutdown_connections 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.852+0000 7f3fa2081740 1 -- 172.21.15.129:0/3772613316 wait complete. 2024-04-26T21:20:16.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.852+0000 7f3fa2081740 20 client.25226 trim_cache size 0 max 16384 2024-04-26T21:20:16.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3fa2081740 1 Processor -- start 2024-04-26T21:20:16.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3fa2081740 1 -- start start 2024-04-26T21:20:16.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f2e10 0x55b2e88f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 0x55b2e890d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f49d0 0x55b2e88ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3fa2081740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f981a5c50 con 0x55b2e88f2e10 2024-04-26T21:20:16.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3fa2081740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f981a84d0 con 0x55b2e88f49d0 2024-04-26T21:20:16.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3fa2081740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3f981a62c0 con 0x55b2e88f4030 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d106640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f2e10 0x55b2e88f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d106640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f2e10 0x55b2e88f65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51888/0 (socket says 172.21.15.129:51888) 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d106640 1 -- 172.21.15.129:0/2284423418 learned_addr learned my addr 172.21.15.129:0/2284423418 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d907640 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f49d0 0x55b2e88ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9c905640 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 0x55b2e890d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d106640 1 -- 172.21.15.129:0/2284423418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 msgr2=0x55b2e890d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d106640 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 0x55b2e890d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d106640 1 -- 172.21.15.129:0/2284423418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f49d0 msgr2=0x55b2e88ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d106640 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f49d0 0x55b2e88ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d106640 1 -- 172.21.15.129:0/2284423418 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e85290f0 con 0x55b2e88f2e10 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.870+0000 7f3f9d907640 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f49d0 0x55b2e88ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.871+0000 7f3f9d106640 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f2e10 0x55b2e88f65f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3f88003e40 tx=0x7f3f88004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a11492c2199292 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.871+0000 7f3f82ffd640 1 -- 172.21.15.129:0/2284423418 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880880a0 con 0x55b2e88f2e10 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.871+0000 7f3f82ffd640 1 -- 172.21.15.129:0/2284423418 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f88088240 con 0x55b2e88f2e10 2024-04-26T21:20:16.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.871+0000 7f3f82ffd640 1 -- 172.21.15.129:0/2284423418 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880ce9f0 con 0x55b2e88f2e10 2024-04-26T21:20:16.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 -- 172.21.15.129:0/2284423418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f2e10 msgr2=0x55b2e88f65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f2e10 0x55b2e88f65f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3f88003e40 tx=0x7f3f88004420 comp rx=0 tx=0).stop 2024-04-26T21:20:16.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 -- 172.21.15.129:0/2284423418 shutdown_connections 2024-04-26T21:20:16.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f49d0 0x55b2e88ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f4030 0x55b2e890d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 --2- 172.21.15.129:0/2284423418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f2e10 0x55b2e88f65f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 -- 172.21.15.129:0/2284423418 >> 172.21.15.129:0/2284423418 conn(0x55b2e88ee3d0 msgr2=0x55b2e88f5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 -- 172.21.15.129:0/2284423418 shutdown_connections 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 -- 172.21.15.129:0/2284423418 wait complete. 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 -- start start 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.872+0000 7f3fa2081740 1 Processor -- start 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 0x55b2e8909250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e8909790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3fa2081740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e890c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3fa2081740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f981a6990 con 0x55b2e88f49d0 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3fa2081740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f981a7e00 con 0x55b2e88f4030 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3fa2081740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3f981a7060 con 0x55b2e88f2e10 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3fa2081740 10 client.0 _ll_register_callbacks cb 0x7f3f880046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9d907640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e890c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9c905640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e8909790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9d106640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 0x55b2e8909250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9c905640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e8909790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60750/0 (socket says 172.21.15.129:60750) 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9c905640 1 -- 172.21.15.129:0/1070588920 learned_addr learned my addr 172.21.15.129:0/1070588920 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9c905640 1 -- 172.21.15.129:0/1070588920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 msgr2=0x55b2e8909250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9c905640 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 0x55b2e8909250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9c905640 1 -- 172.21.15.129:0/1070588920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 msgr2=0x55b2e890c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9c905640 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e890c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9c905640 1 -- 172.21.15.129:0/1070588920 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e88184b0 con 0x55b2e88f4030 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9d106640 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 0x55b2e8909250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f9c905640 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e8909790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3f94004010 tx=0x7f3f94015070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdea4775c84de0b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.873+0000 7f3f80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3f80ff9640 1 -- 172.21.15.129:0/1070588920 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94030060 con 0x55b2e88f4030 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3fa2081740 1 -- 172.21.15.129:0/1070588920 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f88088240 con 0x55b2e88f4030 2024-04-26T21:20:16.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3f80ff9640 1 -- 172.21.15.129:0/1070588920 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3f9402a830 con 0x55b2e88f4030 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3f80ff9640 1 -- 172.21.15.129:0/1070588920 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3f9401e420 con 0x55b2e88f4030 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3fa2081740 10 client.25232 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3fa2081740 1 -- 172.21.15.129:0/1070588920 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b2e8977f80 con 0x55b2e88f4030 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3f80ff9640 1 -- 172.21.15.129:0/1070588920 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f88088240 con 0x55b2e88f4030 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3f80ff9640 1 -- 172.21.15.129:0/1070588920 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f3f940364f0 con 0x55b2e88f4030 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3fa2081740 10 client.25232 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3fa2081740 10 client.25232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3fa2081740 20 client.25232 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3fa2081740 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2e8978960 0x55b2e897ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3fa2081740 4 client.25232 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.874+0000 7f3fa2081740 1 -- 172.21.15.129:0/1070588920 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2e885f800 con 0x55b2e8978960 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.875+0000 7f3f9d106640 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2e8978960 0x55b2e897ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.875+0000 7f3f80ff9640 10 client.25232 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:16.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.875+0000 7f3f9d106640 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2e8978960 0x55b2e897ad80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41a99c82c6491be server_cookie=92f956609348bbb2 in_seq=0 out_seq=0 2024-04-26T21:20:16.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3f80ff9640 1 -- 172.21.15.129:0/1070588920 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b2e885f800 con 0x55b2e8978960 2024-04-26T21:20:16.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3f80ff9640 10 client.25232 handle_command_reply: tid=0 2024-04-26T21:20:16.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 client.25232 shutdown 2024-04-26T21:20:16.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 -- 172.21.15.129:0/1070588920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 msgr2=0x55b2e8909790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e8909790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3f94004010 tx=0x7f3f94015070 comp rx=0 tx=0).stop 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 -- 172.21.15.129:0/1070588920 shutdown_connections 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55b2e8978960 0x55b2e897ad80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b2e88f4030 0x55b2e8909790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2e88f49d0 0x55b2e890c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 --2- 172.21.15.129:0/1070588920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b2e88f2e10 0x55b2e8909250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 -- 172.21.15.129:0/1070588920 >> 172.21.15.129:0/1070588920 conn(0x55b2e88ee3d0 msgr2=0x55b2e88eccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3f80ff9640 0 client.25232 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 -- 172.21.15.129:0/1070588920 shutdown_connections 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 1 -- 172.21.15.129:0/1070588920 wait complete. 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.881+0000 7f3fa2081740 20 client.25232 trim_cache size 0 max 16384 2024-04-26T21:20:16.883 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:20:16.884 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:20:16.884 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:20:16.884 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 -- 172.21.15.129:0/400544339 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3f840a3c40 msgr2=0x7f3f840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3f840a3c40 0x7f3f840a6100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f9806b5e0 tx=0x7f3f9400ba60 comp rx=0 tx=0).stop 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 -- 172.21.15.129:0/400544339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f98101870 msgr2=0x7f3f9806b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f98101870 0x7f3f9806b870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800e9c0 tx=0x7f3f8800ee80 comp rx=0 tx=0).stop 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 -- 172.21.15.129:0/400544339 shutdown_connections 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f3f840a3c40 0x7f3f840a6100 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f98103420 0x7f3f9819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f98102a60 0x7f3f9819d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 --2- 172.21.15.129:0/400544339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f98101870 0x7f3f9806b870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 -- 172.21.15.129:0/400544339 >> 172.21.15.129:0/400544339 conn(0x7f3f980fce20 msgr2=0x7f3f98105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 -- 172.21.15.129:0/400544339 shutdown_connections 2024-04-26T21:20:16.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:16.882+0000 7f3f9f36a640 1 -- 172.21.15.129:0/400544339 wait complete. 2024-04-26T21:20:16.886 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:16 smithi150 ceph-mon[32290]: pgmap v418: 97 pgs: 97 active+clean; 158 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.0 MiB/s wr, 557 op/s 2024-04-26T21:20:16.903 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:20:16.903 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:17.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 -- 172.21.15.129:0/684991437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44105d40 msgr2=0x7f2a44108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 --2- 172.21.15.129:0/684991437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44105d40 0x7f2a44108130 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009900 tx=0x7f2a3402a410 comp rx=0 tx=0).stop 2024-04-26T21:20:17.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 -- 172.21.15.129:0/684991437 shutdown_connections 2024-04-26T21:20:17.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 --2- 172.21.15.129:0/684991437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a44108670 0x7f2a4410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 --2- 172.21.15.129:0/684991437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44105d40 0x7f2a44108130 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 --2- 172.21.15.129:0/684991437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a44103410 0x7f2a44105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 -- 172.21.15.129:0/684991437 >> 172.21.15.129:0/684991437 conn(0x7f2a440fce20 msgr2=0x7f2a440ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 -- 172.21.15.129:0/684991437 shutdown_connections 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 -- 172.21.15.129:0/684991437 wait complete. 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 Processor -- start 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.016+0000 7f2a4c225640 1 -- start start 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a4c225640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44103410 0x7f2a44198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a4c225640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a44105d40 0x7f2a441991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a4c225640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a44108670 0x7f2a441a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a4c225640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4410e510 con 0x7f2a44103410 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a4c225640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4410e390 con 0x7f2a44108670 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a4c225640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4410e690 con 0x7f2a44105d40 2024-04-26T21:20:17.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a49fc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44103410 0x7f2a44198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a49fc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44103410 0x7f2a44198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51930/0 (socket says 172.21.15.129:51930) 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a49fc1640 1 -- 172.21.15.129:0/18714724 learned_addr learned my addr 172.21.15.129:0/18714724 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a4a7c2640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a44108670 0x7f2a441a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a497c0640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a44105d40 0x7f2a441991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a49fc1640 1 -- 172.21.15.129:0/18714724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a44105d40 msgr2=0x7f2a441991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a49fc1640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a44105d40 0x7f2a441991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a49fc1640 1 -- 172.21.15.129:0/18714724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a44108670 msgr2=0x7f2a441a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a49fc1640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a44108670 0x7f2a441a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a49fc1640 1 -- 172.21.15.129:0/18714724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a441a0870 con 0x7f2a44103410 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a4a7c2640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a44108670 0x7f2a441a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.017+0000 7f2a49fc1640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44103410 0x7f2a44198c60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000e9f0 tx=0x7f2a4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.018+0000 7f2a3affd640 1 -- 172.21.15.129:0/18714724 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4000cd80 con 0x7f2a44103410 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.018+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a441a0b60 con 0x7f2a44103410 2024-04-26T21:20:17.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.018+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a441a1100 con 0x7f2a44103410 2024-04-26T21:20:17.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.018+0000 7f2a3affd640 1 -- 172.21.15.129:0/18714724 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a400108e0 con 0x7f2a44103410 2024-04-26T21:20:17.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.018+0000 7f2a3affd640 1 -- 172.21.15.129:0/18714724 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400188e0 con 0x7f2a44103410 2024-04-26T21:20:17.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.019+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a440fe910 con 0x7f2a44103410 2024-04-26T21:20:17.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.020+0000 7f2a3affd640 1 -- 172.21.15.129:0/18714724 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2a40010430 con 0x7f2a44103410 2024-04-26T21:20:17.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.021+0000 7f2a3affd640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2a300a3c40 0x7f2a300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.021+0000 7f2a497c0640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2a300a3c40 0x7f2a300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.021+0000 7f2a3affd640 1 -- 172.21.15.129:0/18714724 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f2a400d0aa0 con 0x7f2a44103410 2024-04-26T21:20:17.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.021+0000 7f2a497c0640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2a300a3c40 0x7f2a300a6100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005ab0 tx=0x7f2a34005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:17.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.025+0000 7f2a3affd640 1 -- 172.21.15.129:0/18714724 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2a4009ccc0 con 0x7f2a44103410 2024-04-26T21:20:17.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.335+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a4402dca0 con 0x7f2a44103410 2024-04-26T21:20:17.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.336+0000 7f2a3affd640 1 -- 172.21.15.129:0/18714724 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f2a400a0fc0 con 0x7f2a44103410 2024-04-26T21:20:17.337 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:20:17.337 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:20:17.337 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:20:17.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2a300a3c40 msgr2=0x7f2a300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2a300a3c40 0x7f2a300a6100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005ab0 tx=0x7f2a34005a20 comp rx=0 tx=0).stop 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44103410 msgr2=0x7f2a44198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44103410 0x7f2a44198c60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000e9f0 tx=0x7f2a4000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 shutdown_connections 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f2a300a3c40 0x7f2a300a6100 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a44108670 0x7f2a441a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a44105d40 0x7f2a441991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 --2- 172.21.15.129:0/18714724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44103410 0x7f2a44198c60 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000e9f0 tx=0x7f2a4000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.338+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 >> 172.21.15.129:0/18714724 conn(0x7f2a440fce20 msgr2=0x7f2a44103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.339+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 shutdown_connections 2024-04-26T21:20:17.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.339+0000 7f2a4c225640 1 -- 172.21.15.129:0/18714724 wait complete. 2024-04-26T21:20:17.356 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:20:17.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 -- 172.21.15.129:0/4209872712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 msgr2=0x7f63f0101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 --2- 172.21.15.129:0/4209872712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 0x7f63f0101cb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009900 tx=0x7f63ec02a410 comp rx=0 tx=0).stop 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 -- 172.21.15.129:0/4209872712 shutdown_connections 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 --2- 172.21.15.129:0/4209872712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63f0103440 0x7f63f0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 --2- 172.21.15.129:0/4209872712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f0102a80 0x7f63f0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 --2- 172.21.15.129:0/4209872712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 0x7f63f0101cb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 -- 172.21.15.129:0/4209872712 >> 172.21.15.129:0/4209872712 conn(0x7f63f00fce00 msgr2=0x7f63f00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 -- 172.21.15.129:0/4209872712 shutdown_connections 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 -- 172.21.15.129:0/4209872712 wait complete. 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 Processor -- start 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.689+0000 7f63f80c8640 1 -- start start 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f80c8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 0x7f63f0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f80c8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f0102a80 0x7f63f0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f80c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63f0103440 0x7f63f01a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f80c8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63f010d410 con 0x7f63f0101890 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f80c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63f010d290 con 0x7f63f0103440 2024-04-26T21:20:17.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f80c8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f63f010d590 con 0x7f63f0102a80 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5e64640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 0x7f63f0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5e64640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 0x7f63f0198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51950/0 (socket says 172.21.15.129:51950) 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5e64640 1 -- 172.21.15.129:0/3217033053 learned_addr learned my addr 172.21.15.129:0/3217033053 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f6665640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63f0103440 0x7f63f01a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5663640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f0102a80 0x7f63f0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5e64640 1 -- 172.21.15.129:0/3217033053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f0102a80 msgr2=0x7f63f0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5e64640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f0102a80 0x7f63f0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5e64640 1 -- 172.21.15.129:0/3217033053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63f0103440 msgr2=0x7f63f01a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5e64640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63f0103440 0x7f63f01a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5e64640 1 -- 172.21.15.129:0/3217033053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f01a0900 con 0x7f63f0101890 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f6665640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63f0103440 0x7f63f01a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f5e64640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 0x7f63f0198cf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f63ec006010 tx=0x7f63ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:17.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63e6ffd640 1 -- 172.21.15.129:0/3217033053 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec03f030 con 0x7f63f0101890 2024-04-26T21:20:17.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63e6ffd640 1 -- 172.21.15.129:0/3217033053 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ec0318f0 con 0x7f63f0101890 2024-04-26T21:20:17.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f80c8640 1 -- 172.21.15.129:0/3217033053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f01a0b90 con 0x7f63f0101890 2024-04-26T21:20:17.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63e6ffd640 1 -- 172.21.15.129:0/3217033053 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec035070 con 0x7f63f0101890 2024-04-26T21:20:17.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.690+0000 7f63f80c8640 1 -- 172.21.15.129:0/3217033053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f01a1080 con 0x7f63f0101890 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.693+0000 7f63e6ffd640 1 -- 172.21.15.129:0/3217033053 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f63ec042030 con 0x7f63f0101890 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.693+0000 7f63fade5740 1 Processor -- start 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63fade5740 1 -- start start 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63fade5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63fade5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63fade5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63fade5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55be6210fd80 con 0x55be621b0e00 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63fade5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55be620c9a20 con 0x55be621afc30 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63fade5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55be6210bd40 con 0x55be621b17c0 2024-04-26T21:20:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f5e64640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f6665640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f6665640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60800/0 (socket says 172.21.15.129:60800) 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f6665640 1 -- 172.21.15.129:0/2050584629 learned_addr learned my addr 172.21.15.129:0/2050584629 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f5e64640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51960/0 (socket says 172.21.15.129:51960) 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63e6ffd640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f63dc0a3c40 0x7f63dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f5663640 1 --2- 172.21.15.129:0/2050584629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f6665640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f63dc0a3c40 0x7f63dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f6665640 1 -- 172.21.15.129:0/2050584629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 msgr2=0x55be621b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f6665640 1 --2- 172.21.15.129:0/2050584629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f6665640 1 -- 172.21.15.129:0/2050584629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 msgr2=0x55be621b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f6665640 1 --2- 172.21.15.129:0/2050584629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63e6ffd640 1 -- 172.21.15.129:0/3217033053 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f63ec0f0bf0 con 0x7f63f0101890 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.694+0000 7f63f6665640 1 -- 172.21.15.129:0/2050584629 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be620d4c30 con 0x55be621afc30 2024-04-26T21:20:17.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.695+0000 7f63f6665640 1 --2- 172.21.15.129:0/2050584629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b0050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63e800fe70 tx=0x7f63e8013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98162d30cf8b8ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.695+0000 7f63f6665640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f63dc0a3c40 0x7f63dc0a6100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63e801e000 tx=0x7f63e8020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.695+0000 7f63dbfff640 1 -- 172.21.15.129:0/2050584629 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e800f890 con 0x55be621afc30 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.695+0000 7f63dbfff640 1 -- 172.21.15.129:0/2050584629 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e8023030 con 0x55be621afc30 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.695+0000 7f63dbfff640 1 -- 172.21.15.129:0/2050584629 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8004530 con 0x55be621afc30 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 -- 172.21.15.129:0/2050584629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 msgr2=0x55be621b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 --2- 172.21.15.129:0/2050584629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b0050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63e800fe70 tx=0x7f63e8013830 comp rx=0 tx=0).stop 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 -- 172.21.15.129:0/2050584629 shutdown_connections 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 --2- 172.21.15.129:0/2050584629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 --2- 172.21.15.129:0/2050584629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 --2- 172.21.15.129:0/2050584629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b0050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 -- 172.21.15.129:0/2050584629 >> 172.21.15.129:0/2050584629 conn(0x55be620ec910 msgr2=0x55be621a7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 -- 172.21.15.129:0/2050584629 shutdown_connections 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 -- 172.21.15.129:0/2050584629 wait complete. 2024-04-26T21:20:17.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 -- start start 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 Processor -- start 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621c1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.696+0000 7f63fade5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55be621b8770 con 0x55be621b0e00 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63fade5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55be62096dc0 con 0x55be621afc30 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63fade5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55be621b88f0 con 0x55be621b17c0 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63fade5740 10 client.0 _ll_register_callbacks cb 0x55be620b2600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5663640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5e64640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5663640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621ba4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51972/0 (socket says 172.21.15.129:51972) 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5663640 1 -- 172.21.15.129:0/1658967702 learned_addr learned my addr 172.21.15.129:0/1658967702 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f6665640 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5e64640 1 -- 172.21.15.129:0/1658967702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 msgr2=0x55be621c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:17.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5e64640 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621c1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5e64640 1 -- 172.21.15.129:0/1658967702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 msgr2=0x55be621ba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5e64640 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621ba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5e64640 1 -- 172.21.15.129:0/1658967702 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be61e213f0 con 0x55be621afc30 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5663640 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f6665640 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63f5e64640 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b9f60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0ee590 tx=0x7f63ec0eea10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9453c657bee3112 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63d97fa640 1 -- 172.21.15.129:0/1658967702 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec049070 con 0x55be621afc30 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63fade5740 1 -- 172.21.15.129:0/1658967702 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e8023030 con 0x55be621afc30 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63d97fa640 1 -- 172.21.15.129:0/1658967702 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f63ec0bd1c0 con 0x55be621afc30 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63d97fa640 1 -- 172.21.15.129:0/1658967702 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f63ec0bd460 con 0x55be621afc30 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63fade5740 10 client.25250 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.697+0000 7f63fade5740 1 -- 172.21.15.129:0/1658967702 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55be61eeb7d0 con 0x55be621afc30 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63d97fa640 1 -- 172.21.15.129:0/1658967702 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e8023030 con 0x55be621afc30 2024-04-26T21:20:17.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63d97fa640 1 -- 172.21.15.129:0/1658967702 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f63ec0c5540 con 0x55be621afc30 2024-04-26T21:20:17.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63fade5740 10 client.25250 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:17.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63fade5740 10 client.25250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:17.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63fade5740 20 client.25250 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:17.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63fade5740 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55be621c5cf0 0x55be621c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63fade5740 4 client.25250 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:17.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63fade5740 1 -- 172.21.15.129:0/1658967702 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be6211d990 con 0x55be621c5cf0 2024-04-26T21:20:17.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63f5663640 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55be621c5cf0 0x55be621c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63d97fa640 10 client.25250 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:17.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.698+0000 7f63f5663640 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55be621c5cf0 0x55be621c8160 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee3526e50e4833ac server_cookie=8bef527cea662a37 in_seq=0 out_seq=0 2024-04-26T21:20:17.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63d97fa640 1 -- 172.21.15.129:0/1658967702 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be6211d990 con 0x55be621c5cf0 2024-04-26T21:20:17.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63d97fa640 10 client.25250 handle_command_reply: tid=0 2024-04-26T21:20:17.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 client.25250 shutdown 2024-04-26T21:20:17.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 -- 172.21.15.129:0/1658967702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 msgr2=0x55be621b9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b9f60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0ee590 tx=0x7f63ec0eea10 comp rx=0 tx=0).stop 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 -- 172.21.15.129:0/1658967702 shutdown_connections 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55be621c5cf0 0x55be621c8160 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 --2- 172.21.15.129:0/1658967702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621b9f60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63d97fa640 0 client.25250 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 -- 172.21.15.129:0/1658967702 >> 172.21.15.129:0/1658967702 conn(0x55be620ec910 msgr2=0x55be621a6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 -- 172.21.15.129:0/1658967702 shutdown_connections 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 1 -- 172.21.15.129:0/1658967702 wait complete. 2024-04-26T21:20:17.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.702+0000 7f63fade5740 20 client.25250 trim_cache size 0 max 16384 2024-04-26T21:20:17.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63fade5740 1 Processor -- start 2024-04-26T21:20:17.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63fade5740 1 -- start start 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63fade5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621afc30 0x55be621ba400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63fade5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63fade5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621b17c0 0x55be621aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63fade5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63f01a5bf0 con 0x55be621b0e00 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63fade5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63f01a8470 con 0x55be621b17c0 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63fade5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f63f01a6260 con 0x55be621afc30 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63f5663640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63f5663640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621c1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51980/0 (socket says 172.21.15.129:51980) 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63f5663640 1 -- 172.21.15.129:0/2626120020 learned_addr learned my addr 172.21.15.129:0/2626120020 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63f6665640 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621b17c0 0x55be621aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.716+0000 7f63f5e64640 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621afc30 0x55be621ba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63f5663640 1 -- 172.21.15.129:0/2626120020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621afc30 msgr2=0x55be621ba400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63f5663640 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621afc30 0x55be621ba400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63f5663640 1 -- 172.21.15.129:0/2626120020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621b17c0 msgr2=0x55be621aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63f5663640 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621b17c0 0x55be621aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63f5663640 1 -- 172.21.15.129:0/2626120020 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be621a7370 con 0x55be621b0e00 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63f6665640 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621b17c0 0x55be621aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63f5663640 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621c1430 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f63e0006de0 tx=0x7f63e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0517a55092a93cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63db7fe640 1 -- 172.21.15.129:0/2626120020 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0016940 con 0x55be621b0e00 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63db7fe640 1 -- 172.21.15.129:0/2626120020 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e0020030 con 0x55be621b0e00 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.717+0000 7f63db7fe640 1 -- 172.21.15.129:0/2626120020 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e00243d0 con 0x55be621b0e00 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 -- 172.21.15.129:0/2626120020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 msgr2=0x55be621c1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621c1430 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f63e0006de0 tx=0x7f63e0008040 comp rx=0 tx=0).stop 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 -- 172.21.15.129:0/2626120020 shutdown_connections 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621b17c0 0x55be621aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621c1430 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 --2- 172.21.15.129:0/2626120020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621afc30 0x55be621ba400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 -- 172.21.15.129:0/2626120020 >> 172.21.15.129:0/2626120020 conn(0x55be620ec910 msgr2=0x55be621aa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 -- 172.21.15.129:0/2626120020 shutdown_connections 2024-04-26T21:20:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 -- 172.21.15.129:0/2626120020 wait complete. 2024-04-26T21:20:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 -- start start 2024-04-26T21:20:17.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:17 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/18714724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:20:17.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.718+0000 7f63fade5740 1 Processor -- start 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63fade5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63fade5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63fade5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63fade5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63f01a6930 con 0x55be621b0e00 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63fade5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63f01a7da0 con 0x55be621afc30 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63fade5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f63f01a7000 con 0x55be621b17c0 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63fade5740 10 client.0 _ll_register_callbacks cb 0x7f63ec004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5663640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5e64640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5663640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621bed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51992/0 (socket says 172.21.15.129:51992) 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5e64640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621be820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60832/0 (socket says 172.21.15.129:60832) 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5663640 1 -- 172.21.15.129:0/660890191 learned_addr learned my addr 172.21.15.129:0/660890191 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f6665640 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5663640 1 -- 172.21.15.129:0/660890191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 msgr2=0x55be621bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5663640 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5663640 1 -- 172.21.15.129:0/660890191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 msgr2=0x55be621be820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5663640 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621be820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5663640 1 -- 172.21.15.129:0/660890191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be621af490 con 0x55be621b0e00 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5e64640 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621be820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63f5663640 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621bed60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f63e0004810 tx=0x7f63e0017b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57157db02b890672 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.719+0000 7f63d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63d97fa640 1 -- 172.21.15.129:0/660890191 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0029020 con 0x55be621b0e00 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63d97fa640 1 -- 172.21.15.129:0/660890191 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f63e0016b40 con 0x55be621b0e00 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63fade5740 1 -- 172.21.15.129:0/660890191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e0020030 con 0x55be621b0e00 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63d97fa640 1 -- 172.21.15.129:0/660890191 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f63e0034050 con 0x55be621b0e00 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63fade5740 10 client.15651 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63fade5740 1 -- 172.21.15.129:0/660890191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55be621c12b0 con 0x55be621b0e00 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63d97fa640 1 -- 172.21.15.129:0/660890191 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e0020030 con 0x55be621b0e00 2024-04-26T21:20:17.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63d97fa640 1 -- 172.21.15.129:0/660890191 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f63e0017210 con 0x55be621b0e00 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63fade5740 10 client.15651 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63fade5740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63fade5740 20 client.15651 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63fade5740 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55be62230ca0 0x55be622330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63fade5740 4 client.15651 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63fade5740 1 -- 172.21.15.129:0/660890191 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be62117800 con 0x55be62230ca0 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.720+0000 7f63f5e64640 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55be62230ca0 0x55be622330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.721+0000 7f63d97fa640 10 client.15651 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.721+0000 7f63f5e64640 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55be62230ca0 0x55be622330c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6401ddb7de98ed server_cookie=3ce30feacac89caf in_seq=0 out_seq=0 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63d97fa640 1 -- 172.21.15.129:0/660890191 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be62117800 con 0x55be62230ca0 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63d97fa640 10 client.15651 handle_command_reply: tid=0 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 client.15651 shutdown 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 -- 172.21.15.129:0/660890191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 msgr2=0x55be621bed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621bed60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f63e0004810 tx=0x7f63e0017b10 comp rx=0 tx=0).stop 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 -- 172.21.15.129:0/660890191 shutdown_connections 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55be62230ca0 0x55be622330c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be621b17c0 0x55be621bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be621b0e00 0x55be621bed60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63d97fa640 0 client.15651 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 --2- 172.21.15.129:0/660890191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be621afc30 0x55be621be820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 -- 172.21.15.129:0/660890191 >> 172.21.15.129:0/660890191 conn(0x55be620ec910 msgr2=0x55be621aa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 -- 172.21.15.129:0/660890191 shutdown_connections 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 1 -- 172.21.15.129:0/660890191 wait complete. 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.724+0000 7f63fade5740 20 client.15651 trim_cache size 0 max 16384 2024-04-26T21:20:17.730 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:20:17.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 -- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f63dc0a3c40 msgr2=0x7f63dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f63dc0a3c40 0x7f63dc0a6100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63e801e000 tx=0x7f63e8020000 comp rx=0 tx=0).stop 2024-04-26T21:20:17.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 -- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 msgr2=0x7f63f0198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:17.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 0x7f63f0198cf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f63ec006010 tx=0x7f63ec0042f0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 -- 172.21.15.129:0/3217033053 shutdown_connections 2024-04-26T21:20:17.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f63dc0a3c40 0x7f63dc0a6100 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63f0103440 0x7f63f01a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f0102a80 0x7f63f0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 --2- 172.21.15.129:0/3217033053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f0101890 0x7f63f0198cf0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:17.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 -- 172.21.15.129:0/3217033053 >> 172.21.15.129:0/3217033053 conn(0x7f63f00fce00 msgr2=0x7f63f006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:17.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 -- 172.21.15.129:0/3217033053 shutdown_connections 2024-04-26T21:20:17.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:17.725+0000 7f63f80c8640 1 -- 172.21.15.129:0/3217033053 wait complete. 2024-04-26T21:20:17.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:17 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/18714724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:20:17.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:17 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/18714724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:20:18.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:18 smithi129 ceph-mon[29237]: pgmap v419: 97 pgs: 97 active+clean; 162 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.5 MiB/s wr, 588 op/s 2024-04-26T21:20:18.737 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:20:18.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:18.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:20:18.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:18 smithi171 ceph-mon[33478]: pgmap v419: 97 pgs: 97 active+clean; 162 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.5 MiB/s wr, 588 op/s 2024-04-26T21:20:18.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.843+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3032950551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08108650 msgr2=0x7f9c0810aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.843+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3032950551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08108650 0x7f9c0810aaa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009ca0 tx=0x7f9c0402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.843+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3032950551 shutdown_connections 2024-04-26T21:20:18.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.843+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3032950551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08108650 0x7f9c0810aaa0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.843+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3032950551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c08105d20 0x7f9c08108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.843+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3032950551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c081033f0 0x7f9c081057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.843+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3032950551 >> 172.21.15.129:0/3032950551 conn(0x7f9c080fcde0 msgr2=0x7f9c080ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3032950551 shutdown_connections 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3032950551 wait complete. 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 Processor -- start 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 -- start start 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c081033f0 0x7f9c08198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08105d20 0x7f9c08199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c08108650 0x7f9c081a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0810e4c0 con 0x7f9c08105d20 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0810e340 con 0x7f9c08108650 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0fbc1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0810e640 con 0x7f9c081033f0 2024-04-26T21:20:18.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0d15c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08105d20 0x7f9c08199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0d15c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08105d20 0x7f9c08199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52018/0 (socket says 172.21.15.129:52018) 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0d15c640 1 -- 172.21.15.129:0/3819832064 learned_addr learned my addr 172.21.15.129:0/3819832064 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c08108650 0x7f9c081a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.844+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c081033f0 0x7f9c08198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9c0d15c640 1 -- 172.21.15.129:0/3819832064 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c081033f0 msgr2=0x7f9c08198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c081033f0 0x7f9c08198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9c0d15c640 1 -- 172.21.15.129:0/3819832064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c08108650 msgr2=0x7f9c081a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c08108650 0x7f9c081a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9c0d15c640 1 -- 172.21.15.129:0/3819832064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c081a07e0 con 0x7f9c08105d20 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c08108650 0x7f9c081a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08105d20 0x7f9c08199100 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800d740 tx=0x7f9bf800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9bfeffd640 1 -- 172.21.15.129:0/3819832064 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8017070 con 0x7f9c08105d20 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9bfeffd640 1 -- 172.21.15.129:0/3819832064 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf800de50 con 0x7f9c08105d20 2024-04-26T21:20:18.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3819832064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c081a0ad0 con 0x7f9c08105d20 2024-04-26T21:20:18.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9bfeffd640 1 -- 172.21.15.129:0/3819832064 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8004ce0 con 0x7f9c08105d20 2024-04-26T21:20:18.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.845+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3819832064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c081a1020 con 0x7f9c08105d20 2024-04-26T21:20:18.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.847+0000 7f9bfeffd640 1 -- 172.21.15.129:0/3819832064 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9bf8004e80 con 0x7f9c08105d20 2024-04-26T21:20:18.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c129b5740 1 Processor -- start 2024-04-26T21:20:18.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c129b5740 1 -- start start 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f352bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352c2a0 0x55d8f3530da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f3531410 0x55d8f35338a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c129b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d8f3490b00 con 0x55d8f3531410 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c129b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d8f342c920 con 0x55d8f352b8b0 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c129b5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d8f348cac0 con 0x55d8f352c2a0 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c0e15e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f352bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c0d15c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f3531410 0x55d8f35338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c0e15e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f352bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60856/0 (socket says 172.21.15.129:60856) 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c0e15e640 1 -- 172.21.15.129:0/2056502036 learned_addr learned my addr 172.21.15.129:0/2056502036 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9bfeffd640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9bf40a3cc0 0x7f9bf40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.848+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/2056502036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352c2a0 0x55d8f3530da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9c0e15e640 1 -- 172.21.15.129:0/2056502036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352c2a0 msgr2=0x55d8f3530da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/2056502036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352c2a0 0x55d8f3530da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9c0e15e640 1 -- 172.21.15.129:0/2056502036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f3531410 msgr2=0x55d8f35338a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/2056502036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f3531410 0x55d8f35338a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9c0e15e640 1 -- 172.21.15.129:0/2056502036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f326a7e0 con 0x55d8f352b8b0 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9bf40a3cc0 0x7f9bf40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9bfeffd640 1 -- 172.21.15.129:0/3819832064 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9bf80cfe00 con 0x7f9c08105d20 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/2056502036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f352bcd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c0402ade0 tx=0x7f9c04009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58d6bac28fa4e572 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9bf3fff640 1 -- 172.21.15.129:0/2056502036 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04043040 con 0x55d8f352b8b0 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9bf40a3cc0 0x7f9bf40a6180 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c0404e000 tx=0x7f9c0404f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9bf3fff640 1 -- 172.21.15.129:0/2056502036 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c04036040 con 0x55d8f352b8b0 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.849+0000 7f9bf3fff640 1 -- 172.21.15.129:0/2056502036 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0402fb20 con 0x55d8f352b8b0 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 -- 172.21.15.129:0/2056502036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 msgr2=0x55d8f352bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 --2- 172.21.15.129:0/2056502036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f352bcd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c0402ade0 tx=0x7f9c04009800 comp rx=0 tx=0).stop 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 -- 172.21.15.129:0/2056502036 shutdown_connections 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 --2- 172.21.15.129:0/2056502036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f3531410 0x55d8f35338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 --2- 172.21.15.129:0/2056502036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352c2a0 0x55d8f3530da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 --2- 172.21.15.129:0/2056502036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f352bcd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 -- 172.21.15.129:0/2056502036 >> 172.21.15.129:0/2056502036 conn(0x55d8f3526f20 msgr2=0x55d8f3527320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 -- 172.21.15.129:0/2056502036 shutdown_connections 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 -- 172.21.15.129:0/2056502036 wait complete. 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 -- start start 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.850+0000 7f9c129b5740 1 Processor -- start 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352b8b0 0x55d8f3535d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352c2a0 0x55d8f35362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 0x55d8f353d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c129b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d8f35343c0 con 0x55d8f352b8b0 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c129b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d8f342a5d0 con 0x55d8f352c2a0 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c129b5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d8f3534540 con 0x55d8f3531410 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c129b5740 10 client.0 _ll_register_callbacks cb 0x55d8f3418080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:18.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d95d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352b8b0 0x55d8f3535d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d95d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352b8b0 0x55d8f3535d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52036/0 (socket says 172.21.15.129:52036) 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d95d640 1 -- 172.21.15.129:0/4236748775 learned_addr learned my addr 172.21.15.129:0/4236748775 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352c2a0 0x55d8f35362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d95d640 1 -- 172.21.15.129:0/4236748775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 msgr2=0x55d8f353d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 0x55d8f353d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 0x55d8f353d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d95d640 1 -- 172.21.15.129:0/4236748775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352c2a0 msgr2=0x55d8f35362b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352c2a0 0x55d8f35362b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d95d640 1 -- 172.21.15.129:0/4236748775 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f32caf70 con 0x55d8f352b8b0 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352c2a0 0x55d8f35362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 0x55d8f353d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:18.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352b8b0 0x55d8f3535d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9c00012700 tx=0x7f9c00012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bf82dc775519ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9bf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9bf17fa640 1 -- 172.21.15.129:0/4236748775 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00004050 con 0x55d8f352b8b0 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9bf17fa640 1 -- 172.21.15.129:0/4236748775 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9c000041f0 con 0x55d8f352b8b0 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.851+0000 7f9c129b5740 1 -- 172.21.15.129:0/4236748775 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c04036040 con 0x55d8f352b8b0 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9bf17fa640 1 -- 172.21.15.129:0/4236748775 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9c000043f0 con 0x55d8f352b8b0 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c129b5740 10 client.15663 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c129b5740 1 -- 172.21.15.129:0/4236748775 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d8f3162170 con 0x55d8f352b8b0 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9bf17fa640 1 -- 172.21.15.129:0/4236748775 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c04036040 con 0x55d8f352b8b0 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9bf17fa640 1 -- 172.21.15.129:0/4236748775 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f9c00003ad0 con 0x55d8f352b8b0 2024-04-26T21:20:18.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c129b5740 10 client.15663 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:18.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c129b5740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:18.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c129b5740 20 client.15663 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:18.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c129b5740 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55d8f3541920 0x55d8f3543de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c129b5740 4 client.15663 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:18.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c129b5740 1 -- 172.21.15.129:0/4236748775 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8f349e710 con 0x55d8f3541920 2024-04-26T21:20:18.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55d8f3541920 0x55d8f3543de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9bf17fa640 10 client.15663 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:18.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.852+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55d8f3541920 0x55d8f3543de0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c975f487983d64ce server_cookie=bae0b9d1040e764b in_seq=0 out_seq=0 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.855+0000 7f9bf17fa640 1 -- 172.21.15.129:0/4236748775 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8f349e710 con 0x55d8f3541920 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.855+0000 7f9bf17fa640 10 client.15663 handle_command_reply: tid=0 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.855+0000 7f9c129b5740 1 client.15663 shutdown 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.855+0000 7f9c129b5740 1 -- 172.21.15.129:0/4236748775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352b8b0 msgr2=0x55d8f3535d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.855+0000 7f9c129b5740 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352b8b0 0x55d8f3535d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9c00012700 tx=0x7f9c00012bc0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9c129b5740 1 -- 172.21.15.129:0/4236748775 shutdown_connections 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9c129b5740 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55d8f3541920 0x55d8f3543de0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9c129b5740 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352c2a0 0x55d8f35362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9c129b5740 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 0x55d8f353d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9bf17fa640 0 client.15663 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9c129b5740 1 --2- 172.21.15.129:0/4236748775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352b8b0 0x55d8f3535d70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9c129b5740 1 -- 172.21.15.129:0/4236748775 >> 172.21.15.129:0/4236748775 conn(0x55d8f3526f20 msgr2=0x55d8f35260e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:18.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9c129b5740 1 -- 172.21.15.129:0/4236748775 shutdown_connections 2024-04-26T21:20:18.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9c129b5740 1 -- 172.21.15.129:0/4236748775 wait complete. 2024-04-26T21:20:18.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.856+0000 7f9c129b5740 20 client.15663 trim_cache size 0 max 16384 2024-04-26T21:20:18.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c129b5740 1 Processor -- start 2024-04-26T21:20:18.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c129b5740 1 -- start start 2024-04-26T21:20:18.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352b8b0 0x55d8f3525ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f3531410 0x55d8f35496b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c129b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081a7c80 con 0x55d8f352c2a0 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c129b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081aa500 con 0x55d8f3531410 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c129b5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081a82f0 con 0x55d8f352b8b0 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c0d15c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c0d15c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52040/0 (socket says 172.21.15.129:52040) 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c0d15c640 1 -- 172.21.15.129:0/2514415700 learned_addr learned my addr 172.21.15.129:0/2514415700 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.873+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f3531410 0x55d8f35496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9c0d15c640 1 -- 172.21.15.129:0/2514415700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352b8b0 msgr2=0x55d8f3525ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352b8b0 0x55d8f3525ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352b8b0 0x55d8f3525ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9c0d15c640 1 -- 172.21.15.129:0/2514415700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f3531410 msgr2=0x55d8f35496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f3531410 0x55d8f35496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9c0d15c640 1 -- 172.21.15.129:0/2514415700 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f3526d50 con 0x55d8f352c2a0 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352b8b0 0x55d8f3525ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353d2b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8098480 tx=0x7f9bf80cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbce4a1b1990f068 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9bf37fe640 1 -- 172.21.15.129:0/2514415700 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8020040 con 0x55d8f352c2a0 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9bf37fe640 1 -- 172.21.15.129:0/2514415700 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf809c020 con 0x55d8f352c2a0 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.874+0000 7f9bf37fe640 1 -- 172.21.15.129:0/2514415700 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8098690 con 0x55d8f352c2a0 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 -- 172.21.15.129:0/2514415700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 msgr2=0x55d8f353d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353d2b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8098480 tx=0x7f9bf80cf040 comp rx=0 tx=0).stop 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 -- 172.21.15.129:0/2514415700 shutdown_connections 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f3531410 0x55d8f35496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353d2b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 --2- 172.21.15.129:0/2514415700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f352b8b0 0x55d8f3525ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 -- 172.21.15.129:0/2514415700 >> 172.21.15.129:0/2514415700 conn(0x55d8f3526f20 msgr2=0x55d8f352a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:18.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 -- 172.21.15.129:0/2514415700 shutdown_connections 2024-04-26T21:20:18.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 -- 172.21.15.129:0/2514415700 wait complete. 2024-04-26T21:20:18.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 -- start start 2024-04-26T21:20:18.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 Processor -- start 2024-04-26T21:20:18.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f353b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 0x55d8f353c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081a89c0 con 0x55d8f352c2a0 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081a9e30 con 0x55d8f352b8b0 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081a9090 con 0x55d8f3531410 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.875+0000 7f9c129b5740 10 client.0 _ll_register_callbacks cb 0x7f9c00009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d15c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d95d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f353b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d95d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f353b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60890/0 (socket says 172.21.15.129:60890) 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d95d640 1 -- 172.21.15.129:0/3381635613 learned_addr learned my addr 172.21.15.129:0/3381635613 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d15c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52054/0 (socket says 172.21.15.129:52054) 2024-04-26T21:20:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0e15e640 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 0x55d8f353c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d15c640 1 -- 172.21.15.129:0/3381635613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 msgr2=0x55d8f353c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 0x55d8f353c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d15c640 1 -- 172.21.15.129:0/3381635613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 msgr2=0x55d8f353b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f353b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d15c640 1 -- 172.21.15.129:0/3381635613 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f353cbb0 con 0x55d8f352c2a0 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f353b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c0d15c640 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353b5b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55d8f35311d0 tx=0x7f9bf80a0390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2d7bf4c996a2a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9bf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9bf17fa640 1 -- 172.21.15.129:0/3381635613 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf80b9020 con 0x55d8f352c2a0 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9bf17fa640 1 -- 172.21.15.129:0/3381635613 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9bf8098690 con 0x55d8f352c2a0 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c129b5740 1 -- 172.21.15.129:0/3381635613 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf809c020 con 0x55d8f352c2a0 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9bf17fa640 1 -- 172.21.15.129:0/3381635613 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9bf8020040 con 0x55d8f352c2a0 2024-04-26T21:20:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c129b5740 10 client.15669 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9c129b5740 1 -- 172.21.15.129:0/3381635613 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d8f35a8510 con 0x55d8f352c2a0 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.876+0000 7f9bf17fa640 1 -- 172.21.15.129:0/3381635613 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf809c020 con 0x55d8f352c2a0 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9bf17fa640 1 -- 172.21.15.129:0/3381635613 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f9bf809aee0 con 0x55d8f352c2a0 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9c129b5740 10 client.15669 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9c129b5740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9c129b5740 20 client.15669 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9c129b5740 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55d8f35a8f60 0x55d8f35ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9c129b5740 4 client.15669 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9c129b5740 1 -- 172.21.15.129:0/3381635613 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d8f3498580 con 0x55d8f35a8f60 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55d8f35a8f60 0x55d8f35ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9bf17fa640 10 client.15669 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:18.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:18.877+0000 7f9c0d95d640 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55d8f35a8f60 0x55d8f35ab380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91e09546178c198 server_cookie=69d0568f3285e71e in_seq=0 out_seq=0 2024-04-26T21:20:18.886 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:18 smithi150 ceph-mon[32290]: pgmap v419: 97 pgs: 97 active+clean; 162 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.5 MiB/s wr, 588 op/s 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9bf17fa640 1 -- 172.21.15.129:0/3381635613 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d8f3498580 con 0x55d8f35a8f60 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9bf17fa640 10 client.15669 handle_command_reply: tid=0 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 client.15669 shutdown 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 -- 172.21.15.129:0/3381635613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 msgr2=0x55d8f353b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353b5b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55d8f35311d0 tx=0x7f9bf80a0390 comp rx=0 tx=0).stop 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 -- 172.21.15.129:0/3381635613 shutdown_connections 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55d8f35a8f60 0x55d8f35ab380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8f3531410 0x55d8f353c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8f352c2a0 0x55d8f353b5b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9bf17fa640 0 client.15669 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:19.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 --2- 172.21.15.129:0/3381635613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8f352b8b0 0x55d8f353b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 -- 172.21.15.129:0/3381635613 >> 172.21.15.129:0/3381635613 conn(0x55d8f3526f20 msgr2=0x55d8f35486a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 -- 172.21.15.129:0/3381635613 shutdown_connections 2024-04-26T21:20:19.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 1 -- 172.21.15.129:0/3381635613 wait complete. 2024-04-26T21:20:19.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.215+0000 7f9c129b5740 20 client.15669 trim_cache size 0 max 16384 2024-04-26T21:20:19.218 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:20:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:20:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "fa806bac-8664-49eb-b26b-0d74d04a3bc9", 2024-04-26T21:20:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:20:19.218 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:20:19.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9bf40a3cc0 msgr2=0x7f9bf40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9bf40a3cc0 0x7f9bf40a6180 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c0404e000 tx=0x7f9c0404f000 comp rx=0 tx=0).stop 2024-04-26T21:20:19.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08105d20 msgr2=0x7f9c08199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08105d20 0x7f9c08199100 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800d740 tx=0x7f9bf800dc00 comp rx=0 tx=0).stop 2024-04-26T21:20:19.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3819832064 shutdown_connections 2024-04-26T21:20:19.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9bf40a3cc0 0x7f9bf40a6180 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c08108650 0x7f9c081a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c08105d20 0x7f9c08199100 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 --2- 172.21.15.129:0/3819832064 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c081033f0 0x7f9c08198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3819832064 >> 172.21.15.129:0/3819832064 conn(0x7f9c080fcde0 msgr2=0x7f9c08103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3819832064 shutdown_connections 2024-04-26T21:20:19.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.216+0000 7f9c0fbc1640 1 -- 172.21.15.129:0/3819832064 wait complete. 2024-04-26T21:20:19.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:20:19.243 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.355+0000 7fc48b493640 1 -- 172.21.15.129:0/1007596233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484102a50 msgr2=0x7fc484102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.355+0000 7fc48b493640 1 --2- 172.21.15.129:0/1007596233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484102a50 0x7fc484102ed0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc474009900 tx=0x7fc47402a410 comp rx=0 tx=0).stop 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.355+0000 7fc48b493640 1 -- 172.21.15.129:0/1007596233 shutdown_connections 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.355+0000 7fc48b493640 1 --2- 172.21.15.129:0/1007596233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc484103410 0x7fc484109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.355+0000 7fc48b493640 1 --2- 172.21.15.129:0/1007596233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484102a50 0x7fc484102ed0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.355+0000 7fc48b493640 1 --2- 172.21.15.129:0/1007596233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc484101860 0x7fc484101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.355+0000 7fc48b493640 1 -- 172.21.15.129:0/1007596233 >> 172.21.15.129:0/1007596233 conn(0x7fc4840fcdf0 msgr2=0x7fc4840ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.355+0000 7fc48b493640 1 -- 172.21.15.129:0/1007596233 shutdown_connections 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.355+0000 7fc48b493640 1 -- 172.21.15.129:0/1007596233 wait complete. 2024-04-26T21:20:19.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48b493640 1 Processor -- start 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48b493640 1 -- start start 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48b493640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc484101860 0x7fc484198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48b493640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc484102a50 0x7fc484199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48b493640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484103410 0x7fc4841a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48b493640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc48410d390 con 0x7fc484103410 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48b493640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc48410d210 con 0x7fc484102a50 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48b493640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc48410d510 con 0x7fc484101860 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc489a30640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484103410 0x7fc4841a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc489a30640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484103410 0x7fc4841a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52068/0 (socket says 172.21.15.129:52068) 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48922f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc484101860 0x7fc484198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc489a30640 1 -- 172.21.15.129:0/1599520643 learned_addr learned my addr 172.21.15.129:0/1599520643 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc488a2e640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc484102a50 0x7fc484199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc489a30640 1 -- 172.21.15.129:0/1599520643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc484101860 msgr2=0x7fc484198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc489a30640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc484101860 0x7fc484198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc489a30640 1 -- 172.21.15.129:0/1599520643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc484102a50 msgr2=0x7fc484199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc489a30640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc484102a50 0x7fc484199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc489a30640 1 -- 172.21.15.129:0/1599520643 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4841a0800 con 0x7fc484103410 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.356+0000 7fc48922f640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc484101860 0x7fc484198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.357+0000 7fc489a30640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484103410 0x7fc4841a0190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00eb10 tx=0x7fc47c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.357+0000 7fc47a7fc640 1 -- 172.21.15.129:0/1599520643 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c004b20 con 0x7fc484103410 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.357+0000 7fc47a7fc640 1 -- 172.21.15.129:0/1599520643 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc47c004cc0 con 0x7fc484103410 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.357+0000 7fc48b493640 1 -- 172.21.15.129:0/1599520643 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4841a0af0 con 0x7fc484103410 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.357+0000 7fc47a7fc640 1 -- 172.21.15.129:0/1599520643 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c010670 con 0x7fc484103410 2024-04-26T21:20:19.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.357+0000 7fc48b493640 1 -- 172.21.15.129:0/1599520643 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4841a0fc0 con 0x7fc484103410 2024-04-26T21:20:19.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.359+0000 7fc47a7fc640 1 -- 172.21.15.129:0/1599520643 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc47c005020 con 0x7fc484103410 2024-04-26T21:20:19.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.360+0000 7fc48e1b5740 1 Processor -- start 2024-04-26T21:20:19.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.360+0000 7fc48e1b5740 1 -- start start 2024-04-26T21:20:19.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.360+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17af6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.360+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.360+0000 7fc47a7fc640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc4700a3c40 0x7fc4700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.360+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af7a70 0x55de17afe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.360+0000 7fc48e1b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55de17a5ad80 con 0x55de17af7a70 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.360+0000 7fc48e1b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de17a148b0 con 0x55de17af70b0 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.360+0000 7fc48e1b5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55de17a56d40 con 0x55de17af5ee0 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc489a30640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af7a70 0x55de17afe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc47a7fc640 1 -- 172.21.15.129:0/1599520643 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc47c0d0b00 con 0x7fc484103410 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc48922f640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc4700a3c40 0x7fc4700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc489a30640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af7a70 0x55de17afe300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52076/0 (socket says 172.21.15.129:52076) 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc489a30640 1 -- 172.21.15.129:0/1013849721 learned_addr learned my addr 172.21.15.129:0/1013849721 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc488a2e640 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc488a2e640 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17af6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc489a30640 1 -- 172.21.15.129:0/1013849721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 msgr2=0x55de17af6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc489a30640 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17af6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc489a30640 1 -- 172.21.15.129:0/1013849721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 msgr2=0x55de17af7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc489a30640 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc489a30640 1 -- 172.21.15.129:0/1013849721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de179e5a40 con 0x55de17af7a70 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc488a2e640 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc48922f640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc4700a3c40 0x7fc4700a6100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc4841028b0 tx=0x7fc480005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc489a30640 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af7a70 0x55de17afe300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0ce4d0 tx=0x7fc47c007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9168b44d9c8a2799 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc46f7fe640 1 -- 172.21.15.129:0/1013849721 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c09ca40 con 0x55de17af7a70 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc46f7fe640 1 -- 172.21.15.129:0/1013849721 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc47c09d370 con 0x55de17af7a70 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.361+0000 7fc46f7fe640 1 -- 172.21.15.129:0/1013849721 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0a27b0 con 0x55de17af7a70 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.362+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1013849721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af7a70 msgr2=0x55de17afe300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.362+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af7a70 0x55de17afe300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0ce4d0 tx=0x7fc47c007850 comp rx=0 tx=0).stop 2024-04-26T21:20:19.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.362+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1013849721 shutdown_connections 2024-04-26T21:20:19.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.362+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.362+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af7a70 0x55de17afe300 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.362+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/1013849721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17af6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.363+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1013849721 >> 172.21.15.129:0/1013849721 conn(0x55de17af13d0 msgr2=0x55de17af17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.363+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1013849721 shutdown_connections 2024-04-26T21:20:19.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.363+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1013849721 wait complete. 2024-04-26T21:20:19.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.363+0000 7fc48e1b5740 1 -- start start 2024-04-26T21:20:19.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.363+0000 7fc48e1b5740 1 Processor -- start 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.364+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b00310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.364+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b00850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.364+0000 7fc48922f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b00310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.364+0000 7fc488a2e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b00850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.364+0000 7fc48922f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b00310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:38326/0 (socket says 172.21.15.129:38326) 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.364+0000 7fc488a2e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b00850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52086/0 (socket says 172.21.15.129:52086) 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.364+0000 7fc48922f640 1 -- 172.21.15.129:0/2024505969 learned_addr learned my addr 172.21.15.129:0/2024505969 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.364+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b07890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55de17afea80 con 0x55de17af70b0 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de179e1d10 con 0x55de17af7a70 2024-04-26T21:20:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55de17afec00 con 0x55de17af5ee0 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc48e1b5740 10 client.0 _ll_register_callbacks cb 0x55de17a0e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc489a30640 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b07890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc489a30640 1 -- 172.21.15.129:0/2024505969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 msgr2=0x55de17b00310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc489a30640 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b00310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc489a30640 1 -- 172.21.15.129:0/2024505969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 msgr2=0x55de17b00850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc489a30640 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b00850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc489a30640 1 -- 172.21.15.129:0/2024505969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de178367d0 con 0x55de17af7a70 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc488a2e640 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b00850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc489a30640 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b07890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0ce4d0 tx=0x7fc47c09ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121d2b0aa97524c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc46cff9640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.365+0000 7fc46cff9640 1 -- 172.21.15.129:0/2024505969 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c07b070 con 0x55de17af7a70 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc46cff9640 1 -- 172.21.15.129:0/2024505969 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc47c09af70 con 0x55de17af7a70 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc47c09d370 con 0x55de17af7a70 2024-04-26T21:20:19.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc46cff9640 1 -- 172.21.15.129:0/2024505969 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc47c09c5d0 con 0x55de17af7a70 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc48e1b5740 10 client.25286 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55de1776c3f0 con 0x55de17af7a70 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc46cff9640 1 -- 172.21.15.129:0/2024505969 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc47c0ce6f0 con 0x55de17af7a70 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc46cff9640 1 -- 172.21.15.129:0/2024505969 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fc47c09ced0 con 0x55de17af7a70 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc48e1b5740 10 client.25286 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc48e1b5740 10 client.25286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc48e1b5740 20 client.25286 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17b0b9d0 0x55de17b0ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.366+0000 7fc48e1b5740 4 client.25286 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:19.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.367+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc47c09d370 con 0x55de17b0b9d0 2024-04-26T21:20:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.504+0000 7fc48922f640 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17b0b9d0 0x55de17b0ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.511+0000 7fc46cff9640 10 client.25286 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:19.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.511+0000 7fc48922f640 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17b0b9d0 0x55de17b0ddc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4407ab6d96f1e288 server_cookie=6db9baff42624381 in_seq=0 out_seq=0 2024-04-26T21:20:19.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.514+0000 7fc46cff9640 1 -- 172.21.15.129:0/2024505969 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc47c09d370 con 0x55de17b0b9d0 2024-04-26T21:20:19.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.514+0000 7fc46cff9640 10 client.25286 handle_command_reply: tid=0 2024-04-26T21:20:19.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.514+0000 7fc48e1b5740 1 client.25286 shutdown 2024-04-26T21:20:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.517+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 msgr2=0x55de17b07890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.517+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b07890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0ce4d0 tx=0x7fc47c09ca40 comp rx=0 tx=0).stop 2024-04-26T21:20:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.517+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 shutdown_connections 2024-04-26T21:20:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.517+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17b0b9d0 0x55de17b0ddc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.517+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b07890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.517+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b00850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.517+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/2024505969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b00310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.517+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 >> 172.21.15.129:0/2024505969 conn(0x55de17af13d0 msgr2=0x55de17af0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.517+0000 7fc46cff9640 0 client.25286 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:19.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.518+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 shutdown_connections 2024-04-26T21:20:19.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.518+0000 7fc48e1b5740 1 -- 172.21.15.129:0/2024505969 wait complete. 2024-04-26T21:20:19.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.518+0000 7fc48e1b5740 20 client.25286 trim_cache size 0 max 16384 2024-04-26T21:20:19.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:19 smithi129 ceph-mon[29237]: pgmap v420: 97 pgs: 97 active+clean; 166 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 605 op/s 2024-04-26T21:20:19.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:19 smithi171 ceph-mon[33478]: pgmap v420: 97 pgs: 97 active+clean; 166 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 605 op/s 2024-04-26T21:20:19.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.747+0000 7fc48e1b5740 1 Processor -- start 2024-04-26T21:20:19.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.747+0000 7fc48e1b5740 1 -- start start 2024-04-26T21:20:19.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.747+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af5ee0 0x55de17aec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.748+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.748+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af7a70 0x55de17aff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.748+0000 7fc48e1b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841a5b40 con 0x55de17af5ee0 2024-04-26T21:20:19.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.748+0000 7fc48e1b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841a83a0 con 0x55de17af70b0 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.748+0000 7fc488a2e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.748+0000 7fc48e1b5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841a6190 con 0x55de17af7a70 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.748+0000 7fc488a2e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60916/0 (socket says 172.21.15.129:60916) 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.748+0000 7fc488a2e640 1 -- 172.21.15.129:0/1857777764 learned_addr learned my addr 172.21.15.129:0/1857777764 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.748+0000 7fc489a30640 1 --2- 172.21.15.129:0/1857777764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af7a70 0x55de17aff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.749+0000 7fc488a2e640 1 -- 172.21.15.129:0/1857777764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af7a70 msgr2=0x55de17aff000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.749+0000 7fc488a2e640 1 --2- 172.21.15.129:0/1857777764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af7a70 0x55de17aff000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.749+0000 7fc488a2e640 1 -- 172.21.15.129:0/1857777764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af5ee0 msgr2=0x55de17aec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.749+0000 7fc488a2e640 1 --2- 172.21.15.129:0/1857777764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af5ee0 0x55de17aec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.749+0000 7fc488a2e640 1 -- 172.21.15.129:0/1857777764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de17b0e130 con 0x55de17af70b0 2024-04-26T21:20:19.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.749+0000 7fc488a2e640 1 --2- 172.21.15.129:0/1857777764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af4a90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc47402a930 tx=0x7fc474009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c9291adc312867 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.762+0000 7fc46effd640 1 -- 172.21.15.129:0/1857777764 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47404c070 con 0x55de17af70b0 2024-04-26T21:20:19.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.762+0000 7fc46effd640 1 -- 172.21.15.129:0/1857777764 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc474031e50 con 0x55de17af70b0 2024-04-26T21:20:19.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.762+0000 7fc46effd640 1 -- 172.21.15.129:0/1857777764 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474035040 con 0x55de17af70b0 2024-04-26T21:20:19.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.763+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1857777764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 msgr2=0x55de17af4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.763+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/1857777764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af4a90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc47402a930 tx=0x7fc474009260 comp rx=0 tx=0).stop 2024-04-26T21:20:19.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.764+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1857777764 shutdown_connections 2024-04-26T21:20:19.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.764+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/1857777764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af70b0 0x55de17af4a90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.764+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/1857777764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af7a70 0x55de17aff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.764+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/1857777764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af5ee0 0x55de17aec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.764+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1857777764 >> 172.21.15.129:0/1857777764 conn(0x55de17af13d0 msgr2=0x55de17af4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.768+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1857777764 shutdown_connections 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.768+0000 7fc48e1b5740 1 -- 172.21.15.129:0/1857777764 wait complete. 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.768+0000 7fc48e1b5740 1 -- start start 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.769+0000 7fc48e1b5740 1 Processor -- start 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.769+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b04e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc48e1b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b05dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc48e1b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841a6860 con 0x55de17af70b0 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc48e1b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841a7cd0 con 0x55de17af7a70 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc488a2e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b04e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc48e1b5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841a6f30 con 0x55de17af5ee0 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc489a30640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b05dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc48e1b5740 10 client.0 _ll_register_callbacks cb 0x7fc47c0a5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc489a30640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b05dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60924/0 (socket says 172.21.15.129:60924) 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc489a30640 1 -- 172.21.15.129:0/3580225547 learned_addr learned my addr 172.21.15.129:0/3580225547 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.770+0000 7fc48922f640 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.771+0000 7fc48922f640 1 -- 172.21.15.129:0/3580225547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 msgr2=0x55de17b05dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.771+0000 7fc48922f640 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b05dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.771+0000 7fc48922f640 1 -- 172.21.15.129:0/3580225547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 msgr2=0x55de17b04e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.771+0000 7fc48922f640 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b04e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.771+0000 7fc48922f640 1 -- 172.21.15.129:0/3580225547 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de1772c0f0 con 0x55de17af5ee0 2024-04-26T21:20:19.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.771+0000 7fc48922f640 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b048c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55de17a5aed0 tx=0x7fc480004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b37d343a73edcdb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.787+0000 7fc46cff9640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-26T21:20:19.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.787+0000 7fc46cff9640 1 -- 172.21.15.129:0/3580225547 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48002a440 con 0x55de17af5ee0 2024-04-26T21:20:19.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.787+0000 7fc46cff9640 1 -- 172.21.15.129:0/3580225547 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc48002a5e0 con 0x55de17af5ee0 2024-04-26T21:20:19.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.787+0000 7fc48e1b5740 1 -- 172.21.15.129:0/3580225547 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc474031e50 con 0x55de17af5ee0 2024-04-26T21:20:19.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.797+0000 7fc46cff9640 1 -- 172.21.15.129:0/3580225547 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc48002a7e0 con 0x55de17af5ee0 2024-04-26T21:20:19.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.797+0000 7fc48e1b5740 10 client.25219 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:19.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.797+0000 7fc48e1b5740 1 -- 172.21.15.129:0/3580225547 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55de17a1b4b0 con 0x55de17af5ee0 2024-04-26T21:20:19.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.801+0000 7fc46cff9640 1 -- 172.21.15.129:0/3580225547 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc474031e50 con 0x55de17af5ee0 2024-04-26T21:20:19.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.801+0000 7fc46cff9640 1 -- 172.21.15.129:0/3580225547 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fc480028860 con 0x55de17af5ee0 2024-04-26T21:20:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.802+0000 7fc48e1b5740 10 client.25219 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.802+0000 7fc48e1b5740 10 client.25219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.802+0000 7fc48e1b5740 20 client.25219 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.802+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17b72850 0x55de17b74c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.802+0000 7fc48e1b5740 4 client.25219 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:20:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.802+0000 7fc48e1b5740 1 -- 172.21.15.129:0/3580225547 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55de17a68990 con 0x55de17b72850 2024-04-26T21:20:19.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.802+0000 7fc488a2e640 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17b72850 0x55de17b74c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.803+0000 7fc488a2e640 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17b72850 0x55de17b74c40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1369d2a4c362f09 server_cookie=9d0fc63779001f31 in_seq=0 out_seq=0 2024-04-26T21:20:19.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.805+0000 7fc46cff9640 10 client.25219 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:19.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.807+0000 7fc46cff9640 1 -- 172.21.15.129:0/3580225547 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55de17a68990 con 0x55de17b72850 2024-04-26T21:20:19.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.807+0000 7fc46cff9640 10 client.25219 handle_command_reply: tid=0 2024-04-26T21:20:19.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.807+0000 7fc48e1b5740 1 client.25219 shutdown 2024-04-26T21:20:19.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.807+0000 7fc48e1b5740 1 -- 172.21.15.129:0/3580225547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 msgr2=0x55de17b048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.807+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b048c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55de17a5aed0 tx=0x7fc480004730 comp rx=0 tx=0).stop 2024-04-26T21:20:19.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc48e1b5740 1 -- 172.21.15.129:0/3580225547 shutdown_connections 2024-04-26T21:20:19.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55de17b72850 0x55de17b74c40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de17af7a70 0x55de17b05dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de17af70b0 0x55de17b04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc46cff9640 0 client.25219 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:19.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc48e1b5740 1 --2- 172.21.15.129:0/3580225547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55de17af5ee0 0x55de17b048c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc48e1b5740 1 -- 172.21.15.129:0/3580225547 >> 172.21.15.129:0/3580225547 conn(0x55de17af13d0 msgr2=0x55de17af4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc48e1b5740 1 -- 172.21.15.129:0/3580225547 shutdown_connections 2024-04-26T21:20:19.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc48e1b5740 1 -- 172.21.15.129:0/3580225547 wait complete. 2024-04-26T21:20:19.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.808+0000 7fc48e1b5740 20 client.25219 trim_cache size 0 max 16384 2024-04-26T21:20:19.810 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:20:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (30 inodes in the stack)", 2024-04-26T21:20:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:20:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "fa806bac-8664-49eb-b26b-0d74d04a3bc9": { 2024-04-26T21:20:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:20:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "fa806bac-8664-49eb-b26b-0d74d04a3bc9", 2024-04-26T21:20:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:20:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:20:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:20:19.812 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:20:19.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.827+0000 7fc48b493640 1 -- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc4700a3c40 msgr2=0x7fc4700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.827+0000 7fc48b493640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc4700a3c40 0x7fc4700a6100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc4841028b0 tx=0x7fc480005f90 comp rx=0 tx=0).stop 2024-04-26T21:20:19.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.827+0000 7fc48b493640 1 -- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484103410 msgr2=0x7fc4841a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.827+0000 7fc48b493640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484103410 0x7fc4841a0190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00eb10 tx=0x7fc47c00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.828+0000 7fc48b493640 1 -- 172.21.15.129:0/1599520643 shutdown_connections 2024-04-26T21:20:19.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.828+0000 7fc48b493640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fc4700a3c40 0x7fc4700a6100 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.828+0000 7fc48b493640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc484102a50 0x7fc484199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.828+0000 7fc48b493640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc484103410 0x7fc4841a0190 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.828+0000 7fc48b493640 1 --2- 172.21.15.129:0/1599520643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc484101860 0x7fc484198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.828+0000 7fc48b493640 1 -- 172.21.15.129:0/1599520643 >> 172.21.15.129:0/1599520643 conn(0x7fc4840fcdf0 msgr2=0x7fc48406af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.837+0000 7fc48b493640 1 -- 172.21.15.129:0/1599520643 shutdown_connections 2024-04-26T21:20:19.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:19.840+0000 7fc48b493640 1 -- 172.21.15.129:0/1599520643 wait complete. 2024-04-26T21:20:19.882 INFO:tasks.cephfs.filesystem:scrub status for tag:fa806bac-8664-49eb-b26b-0d74d04a3bc9 - {'path': '/', 'tag': 'fa806bac-8664-49eb-b26b-0d74d04a3bc9', 'options': 'recursive,force'} 2024-04-26T21:20:19.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:19 smithi150 ceph-mon[32290]: pgmap v420: 97 pgs: 97 active+clean; 166 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 605 op/s 2024-04-26T21:20:20.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:20 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:20:20.723 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:20 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:20:20.723 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:20 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:20:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:20 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:20:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:20 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:20:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:20 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:20:20.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:20 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:20:20.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:20 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:20:20.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:20 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:20:21.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:21 smithi129 ceph-mon[29237]: pgmap v421: 97 pgs: 97 active+clean; 165 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.3 MiB/s wr, 487 op/s 2024-04-26T21:20:21.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:21 smithi171 ceph-mon[33478]: pgmap v421: 97 pgs: 97 active+clean; 165 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.3 MiB/s wr, 487 op/s 2024-04-26T21:20:21.767 INFO:tasks.workunit.client.0.smithi129.stdout: 42 1278064 629 937980 631 545475 1328 2024-04-26T21:20:21.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:21 smithi150 ceph-mon[32290]: pgmap v421: 97 pgs: 97 active+clean; 165 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.3 MiB/s wr, 487 op/s 2024-04-26T21:20:24.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:24 smithi129 ceph-mon[29237]: pgmap v422: 97 pgs: 97 active+clean; 173 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 358 op/s 2024-04-26T21:20:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:24 smithi171 ceph-mon[33478]: pgmap v422: 97 pgs: 97 active+clean; 173 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 358 op/s 2024-04-26T21:20:24.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:24 smithi150 ceph-mon[32290]: pgmap v422: 97 pgs: 97 active+clean; 173 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 358 op/s 2024-04-26T21:20:25.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:25 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:25.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:25 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:25.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:25 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:26 smithi129 ceph-mon[29237]: pgmap v423: 97 pgs: 97 active+clean; 173 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-04-26T21:20:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:20:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:20:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:20:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:20:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:26 smithi171 ceph-mon[33478]: pgmap v423: 97 pgs: 97 active+clean; 173 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-04-26T21:20:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:20:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:20:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:20:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:20:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:26 smithi150 ceph-mon[32290]: pgmap v423: 97 pgs: 97 active+clean; 173 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-04-26T21:20:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:20:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:20:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:20:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:20:27.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:27 smithi129 ceph-mon[29237]: pgmap v424: 97 pgs: 97 active+clean; 173 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-04-26T21:20:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:27 smithi171 ceph-mon[33478]: pgmap v424: 97 pgs: 97 active+clean; 173 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-04-26T21:20:27.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:27 smithi150 ceph-mon[32290]: pgmap v424: 97 pgs: 97 active+clean; 173 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-04-26T21:20:30.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:30 smithi129 ceph-mon[29237]: pgmap v425: 97 pgs: 97 active+clean; 177 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 73 op/s 2024-04-26T21:20:30.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:30 smithi171 ceph-mon[33478]: pgmap v425: 97 pgs: 97 active+clean; 177 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 73 op/s 2024-04-26T21:20:30.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:30 smithi150 ceph-mon[32290]: pgmap v425: 97 pgs: 97 active+clean; 177 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 73 op/s 2024-04-26T21:20:31.767 INFO:tasks.workunit.client.0.smithi129.stdout: 49 940134 472 684693 354 438391 1461 2024-04-26T21:20:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:32 smithi129 ceph-mon[29237]: pgmap v426: 97 pgs: 97 active+clean; 181 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 50 op/s 2024-04-26T21:20:32.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:32 smithi171 ceph-mon[33478]: pgmap v426: 97 pgs: 97 active+clean; 181 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 50 op/s 2024-04-26T21:20:32.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:32 smithi150 ceph-mon[32290]: pgmap v426: 97 pgs: 97 active+clean; 181 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 50 op/s 2024-04-26T21:20:34.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:34 smithi129 ceph-mon[29237]: pgmap v427: 97 pgs: 97 active+clean; 181 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 50 op/s 2024-04-26T21:20:34.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:34 smithi171 ceph-mon[33478]: pgmap v427: 97 pgs: 97 active+clean; 181 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 50 op/s 2024-04-26T21:20:34.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:34 smithi150 ceph-mon[32290]: pgmap v427: 97 pgs: 97 active+clean; 181 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 50 op/s 2024-04-26T21:20:35.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:35 smithi129 ceph-mon[29237]: pgmap v428: 97 pgs: 97 active+clean; 189 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 40 op/s 2024-04-26T21:20:35.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:35 smithi171 ceph-mon[33478]: pgmap v428: 97 pgs: 97 active+clean; 189 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 40 op/s 2024-04-26T21:20:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:35 smithi150 ceph-mon[32290]: pgmap v428: 97 pgs: 97 active+clean; 189 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 40 op/s 2024-04-26T21:20:38.723 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:38 smithi129 ceph-mon[29237]: pgmap v429: 97 pgs: 97 active+clean; 188 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 41 op/s 2024-04-26T21:20:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:38 smithi171 ceph-mon[33478]: pgmap v429: 97 pgs: 97 active+clean; 188 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 41 op/s 2024-04-26T21:20:38.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:38 smithi150 ceph-mon[32290]: pgmap v429: 97 pgs: 97 active+clean; 188 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 41 op/s 2024-04-26T21:20:39.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:39 smithi129 ceph-mon[29237]: pgmap v430: 97 pgs: 97 active+clean; 198 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 74 op/s 2024-04-26T21:20:39.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:39 smithi171 ceph-mon[33478]: pgmap v430: 97 pgs: 97 active+clean; 198 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 74 op/s 2024-04-26T21:20:39.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:39 smithi150 ceph-mon[32290]: pgmap v430: 97 pgs: 97 active+clean; 198 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 74 op/s 2024-04-26T21:20:41.768 INFO:tasks.workunit.client.0.smithi129.stdout: 58 1288869 498 912166 310 614375 882 2024-04-26T21:20:42.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:42 smithi129 ceph-mon[29237]: pgmap v431: 97 pgs: 97 active+clean; 232 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-04-26T21:20:42.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:42 smithi171 ceph-mon[33478]: pgmap v431: 97 pgs: 97 active+clean; 232 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-04-26T21:20:42.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:42 smithi150 ceph-mon[32290]: pgmap v431: 97 pgs: 97 active+clean; 232 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-04-26T21:20:44.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:44 smithi129 ceph-mon[29237]: pgmap v432: 97 pgs: 97 active+clean; 280 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.9 MiB/s wr, 523 op/s 2024-04-26T21:20:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:44 smithi171 ceph-mon[33478]: pgmap v432: 97 pgs: 97 active+clean; 280 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.9 MiB/s wr, 523 op/s 2024-04-26T21:20:44.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:44 smithi150 ceph-mon[32290]: pgmap v432: 97 pgs: 97 active+clean; 280 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.9 MiB/s wr, 523 op/s 2024-04-26T21:20:45.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:45 smithi129 ceph-mon[29237]: pgmap v433: 97 pgs: 97 active+clean; 294 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 624 op/s 2024-04-26T21:20:45.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:45 smithi171 ceph-mon[33478]: pgmap v433: 97 pgs: 97 active+clean; 294 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 624 op/s 2024-04-26T21:20:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:45 smithi150 ceph-mon[32290]: pgmap v433: 97 pgs: 97 active+clean; 294 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 624 op/s 2024-04-26T21:20:48.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:48 smithi129 ceph-mon[29237]: pgmap v434: 97 pgs: 97 active+clean; 292 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.6 MiB/s wr, 651 op/s 2024-04-26T21:20:48.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:48 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:20:48.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:48 smithi171 ceph-mon[33478]: pgmap v434: 97 pgs: 97 active+clean; 292 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.6 MiB/s wr, 651 op/s 2024-04-26T21:20:48.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:48 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:20:48.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:48 smithi150 ceph-mon[32290]: pgmap v434: 97 pgs: 97 active+clean; 292 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.6 MiB/s wr, 651 op/s 2024-04-26T21:20:48.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:48 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:20:49.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:20:49.884 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:20:50.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.354+0000 7f96f046d640 1 -- 172.21.15.129:0/4009095818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96e8102550 msgr2=0x7f96e81029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.354+0000 7f96f046d640 1 --2- 172.21.15.129:0/4009095818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96e8102550 0x7f96e81029d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009be0 tx=0x7f96e002a710 comp rx=0 tx=0).stop 2024-04-26T21:20:50.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.354+0000 7f96f046d640 1 -- 172.21.15.129:0/4009095818 shutdown_connections 2024-04-26T21:20:50.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.354+0000 7f96f046d640 1 --2- 172.21.15.129:0/4009095818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96e8102550 0x7f96e81029d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.354+0000 7f96f046d640 1 --2- 172.21.15.129:0/4009095818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101bb0 0x7f96e8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.354+0000 7f96f046d640 1 --2- 172.21.15.129:0/4009095818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96e8101250 0x7f96e8101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.354+0000 7f96f046d640 1 -- 172.21.15.129:0/4009095818 >> 172.21.15.129:0/4009095818 conn(0x7f96e80fcdf0 msgr2=0x7f96e80ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.359+0000 7f96f046d640 1 -- 172.21.15.129:0/4009095818 shutdown_connections 2024-04-26T21:20:50.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.359+0000 7f96f046d640 1 -- 172.21.15.129:0/4009095818 wait complete. 2024-04-26T21:20:50.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.360+0000 7f96f046d640 1 Processor -- start 2024-04-26T21:20:50.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.361+0000 7f96f046d640 1 -- start start 2024-04-26T21:20:50.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.361+0000 7f96f046d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101250 0x7f96e819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.361+0000 7f96f046d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96e8101bb0 0x7f96e819b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.361+0000 7f96f046d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96e8102550 0x7f96e81a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.361+0000 7f96f046d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8110680 con 0x7f96e8102550 2024-04-26T21:20:50.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96f046d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8110500 con 0x7f96e8101bb0 2024-04-26T21:20:50.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96ee209640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101250 0x7f96e819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96ee209640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101250 0x7f96e819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:42238/0 (socket says 172.21.15.129:42238) 2024-04-26T21:20:50.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96ee209640 1 -- 172.21.15.129:0/2011269754 learned_addr learned my addr 172.21.15.129:0/2011269754 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:50.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96eda08640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96e8101bb0 0x7f96e819b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96f046d640 1 -- 172.21.15.129:0/2011269754 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8110800 con 0x7f96e8101250 2024-04-26T21:20:50.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96eea0a640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96e8102550 0x7f96e81a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96eea0a640 1 -- 172.21.15.129:0/2011269754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101250 msgr2=0x7f96e819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96eea0a640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101250 0x7f96e819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96eea0a640 1 -- 172.21.15.129:0/2011269754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96e8101bb0 msgr2=0x7f96e819b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96eea0a640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96e8101bb0 0x7f96e819b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.362+0000 7f96eea0a640 1 -- 172.21.15.129:0/2011269754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e81a2720 con 0x7f96e8102550 2024-04-26T21:20:50.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.363+0000 7f96eea0a640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96e8102550 0x7f96e81a20b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f96e002ad00 tx=0x7f96e00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.364+0000 7f96df7fe640 1 -- 172.21.15.129:0/2011269754 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0041020 con 0x7f96e8102550 2024-04-26T21:20:50.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.364+0000 7f96df7fe640 1 -- 172.21.15.129:0/2011269754 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e0031920 con 0x7f96e8102550 2024-04-26T21:20:50.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.364+0000 7f96df7fe640 1 -- 172.21.15.129:0/2011269754 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0036040 con 0x7f96e8102550 2024-04-26T21:20:50.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.364+0000 7f96f046d640 1 -- 172.21.15.129:0/2011269754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e81a29b0 con 0x7f96e8102550 2024-04-26T21:20:50.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.364+0000 7f96f046d640 1 -- 172.21.15.129:0/2011269754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e81a2d90 con 0x7f96e8102550 2024-04-26T21:20:50.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.371+0000 7f96f3186740 1 Processor -- start 2024-04-26T21:20:50.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.371+0000 7f96df7fe640 1 -- 172.21.15.129:0/2011269754 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f96e0044020 con 0x7f96e8102550 2024-04-26T21:20:50.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.372+0000 7f96f3186740 1 -- start start 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.372+0000 7f96df7fe640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96d40a3d90 0x7f96d40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.372+0000 7f96f3186740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bb0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96df7fe640 1 -- 172.21.15.129:0/2011269754 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f96e00f1820 con 0x7f96e8102550 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96eda08640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96d40a3d90 0x7f96d40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96f3186740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 0x5615e6bb1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96f3186740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 0x5615e6bb8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96f3186740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5615e6b14d80 con 0x5615e6bafee0 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96f3186740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5615e6ace8b0 con 0x5615e6bb10b0 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96f3186740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5615e6b10d40 con 0x5615e6bb1a70 2024-04-26T21:20:50.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96ee209640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bb0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96ee209640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bb0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40844/0 (socket says 172.21.15.129:40844) 2024-04-26T21:20:50.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96ee209640 1 -- 172.21.15.129:0/752630427 learned_addr learned my addr 172.21.15.129:0/752630427 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:50.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96eea0a640 1 --2- 172.21.15.129:0/752630427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 0x5615e6bb8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96eda08640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96d40a3d90 0x7f96d40a6250 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96e819c010 tx=0x7f96d800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.373+0000 7f96eda08640 1 --2- 172.21.15.129:0/752630427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 0x5615e6bb1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.374+0000 7f96ee209640 1 -- 172.21.15.129:0/752630427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 msgr2=0x5615e6bb8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.374+0000 7f96ee209640 1 --2- 172.21.15.129:0/752630427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 0x5615e6bb8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.374+0000 7f96ee209640 1 -- 172.21.15.129:0/752630427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 msgr2=0x5615e6bb1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.374+0000 7f96ee209640 1 --2- 172.21.15.129:0/752630427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 0x5615e6bb1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.374+0000 7f96ee209640 1 -- 172.21.15.129:0/752630427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e6a9fa40 con 0x5615e6bafee0 2024-04-26T21:20:50.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.374+0000 7f96ee209640 1 --2- 172.21.15.129:0/752630427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bb0300 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009d80 tx=0x7f96e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216da906c731d4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.374+0000 7f96dcbfe640 1 -- 172.21.15.129:0/752630427 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4004230 con 0x5615e6bafee0 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.374+0000 7f96dcbfe640 1 -- 172.21.15.129:0/752630427 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e40043d0 con 0x5615e6bafee0 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.374+0000 7f96dcbfe640 1 -- 172.21.15.129:0/752630427 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e40197e0 con 0x5615e6bafee0 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.375+0000 7f96f3186740 1 -- 172.21.15.129:0/752630427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 msgr2=0x5615e6bb0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.375+0000 7f96f3186740 1 --2- 172.21.15.129:0/752630427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bb0300 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009d80 tx=0x7f96e4011810 comp rx=0 tx=0).stop 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.375+0000 7f96f3186740 1 -- 172.21.15.129:0/752630427 shutdown_connections 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.375+0000 7f96f3186740 1 --2- 172.21.15.129:0/752630427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 0x5615e6bb1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.375+0000 7f96f3186740 1 --2- 172.21.15.129:0/752630427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 0x5615e6bb8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.375+0000 7f96f3186740 1 --2- 172.21.15.129:0/752630427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bb0300 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.375+0000 7f96f3186740 1 -- 172.21.15.129:0/752630427 >> 172.21.15.129:0/752630427 conn(0x5615e6bab3d0 msgr2=0x5615e6bab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.375+0000 7f96f3186740 1 -- 172.21.15.129:0/752630427 shutdown_connections 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.375+0000 7f96f3186740 1 -- 172.21.15.129:0/752630427 wait complete. 2024-04-26T21:20:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.376+0000 7f96f3186740 1 -- start start 2024-04-26T21:20:50.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.376+0000 7f96f3186740 1 Processor -- start 2024-04-26T21:20:50.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96f3186740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96f3186740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 0x5615e6bba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96f3186740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb1a70 0x5615e6bc1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96f3186740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5615e6bb8a80 con 0x5615e6bafee0 2024-04-26T21:20:50.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96f3186740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5615e6a9bd10 con 0x5615e6bb1a70 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96f3186740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5615e6bb8c00 con 0x5615e6bb10b0 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96ee209640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96eea0a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb1a70 0x5615e6bc1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96ee209640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bba3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40852/0 (socket says 172.21.15.129:40852) 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96ee209640 1 -- 172.21.15.129:0/1093270676 learned_addr learned my addr 172.21.15.129:0/1093270676 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96eda08640 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 0x5615e6bba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96f3186740 10 client.0 _ll_register_callbacks cb 0x5615e6ac8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96eea0a640 1 -- 172.21.15.129:0/1093270676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 msgr2=0x5615e6bba900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96eea0a640 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 0x5615e6bba900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.377+0000 7f96eea0a640 1 -- 172.21.15.129:0/1093270676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 msgr2=0x5615e6bba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96eea0a640 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96eea0a640 1 -- 172.21.15.129:0/1093270676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e68f07d0 con 0x5615e6bb1a70 2024-04-26T21:20:50.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96eda08640 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 0x5615e6bba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96ee209640 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96eea0a640 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb1a70 0x5615e6bc1940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96e00ef160 tx=0x7f96e00bd530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e09ddfddb734c710 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96d1ffb640 1 -- 172.21.15.129:0/1093270676 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00bd930 con 0x5615e6bb1a70 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96d1ffb640 1 -- 172.21.15.129:0/1093270676 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f96e00c6110 con 0x5615e6bb1a70 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96f3186740 1 -- 172.21.15.129:0/1093270676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e40043d0 con 0x5615e6bb1a70 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96d1ffb640 1 -- 172.21.15.129:0/1093270676 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f96e00c1ce0 con 0x5615e6bb1a70 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96f3186740 10 client.25301 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.378+0000 7f96f3186740 1 -- 172.21.15.129:0/1093270676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5615e68263f0 con 0x5615e6bb1a70 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.379+0000 7f96d1ffb640 1 -- 172.21.15.129:0/1093270676 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e40043d0 con 0x5615e6bb1a70 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.379+0000 7f96d1ffb640 1 -- 172.21.15.129:0/1093270676 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f96e00dcaf0 con 0x5615e6bb1a70 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.379+0000 7f96f3186740 10 client.25301 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.379+0000 7f96f3186740 10 client.25301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.379+0000 7f96f3186740 20 client.25301 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:50.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.379+0000 7f96f3186740 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5615e6bc5e10 0x5615e6bc8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.379+0000 7f96f3186740 4 client.25301 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:50.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.379+0000 7f96f3186740 1 -- 172.21.15.129:0/1093270676 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615e6b22990 con 0x5615e6bc5e10 2024-04-26T21:20:50.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.379+0000 7f96ee209640 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5615e6bc5e10 0x5615e6bc8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.380+0000 7f96d1ffb640 10 client.25301 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:50.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.380+0000 7f96ee209640 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5615e6bc5e10 0x5615e6bc8290 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b381bcb94dd3e server_cookie=c668e14e859fa833 in_seq=0 out_seq=0 2024-04-26T21:20:50.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96d1ffb640 1 -- 172.21.15.129:0/1093270676 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615e6b22990 con 0x5615e6bc5e10 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96d1ffb640 10 client.25301 handle_command_reply: tid=0 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 client.25301 shutdown 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 -- 172.21.15.129:0/1093270676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb1a70 msgr2=0x5615e6bc1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb1a70 0x5615e6bc1940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96e00ef160 tx=0x7f96e00bd530 comp rx=0 tx=0).stop 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 -- 172.21.15.129:0/1093270676 shutdown_connections 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5615e6bc5e10 0x5615e6bc8290 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb1a70 0x5615e6bc1940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 0x5615e6bba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96d1ffb640 0 client.25301 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 --2- 172.21.15.129:0/1093270676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 -- 172.21.15.129:0/1093270676 >> 172.21.15.129:0/1093270676 conn(0x5615e6bab3d0 msgr2=0x5615e6baa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 -- 172.21.15.129:0/1093270676 shutdown_connections 2024-04-26T21:20:50.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 1 -- 172.21.15.129:0/1093270676 wait complete. 2024-04-26T21:20:50.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.383+0000 7f96f3186740 20 client.25301 trim_cache size 0 max 16384 2024-04-26T21:20:50.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.402+0000 7f96f3186740 1 Processor -- start 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.402+0000 7f96f3186740 1 -- start start 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96f3186740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bafee0 0x5615e6bc1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96f3186740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 0x5615e6ba6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96f3186740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bb1a70 0x5615e6bb4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96f3186740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81a9c60 con 0x5615e6bb1a70 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96f3186740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81ac4e0 con 0x5615e6bafee0 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96f3186740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81aa2d0 con 0x5615e6bb10b0 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eea0a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bb1a70 0x5615e6bb4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96ee209640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bafee0 0x5615e6bc1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eea0a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bb1a70 0x5615e6bb4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40856/0 (socket says 172.21.15.129:40856) 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eea0a640 1 -- 172.21.15.129:0/1732777230 learned_addr learned my addr 172.21.15.129:0/1732777230 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:50.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eda08640 1 --2- 172.21.15.129:0/1732777230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 0x5615e6ba6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eea0a640 1 -- 172.21.15.129:0/1732777230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 msgr2=0x5615e6ba6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eea0a640 1 --2- 172.21.15.129:0/1732777230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 0x5615e6ba6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eea0a640 1 -- 172.21.15.129:0/1732777230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bafee0 msgr2=0x5615e6bc1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eea0a640 1 --2- 172.21.15.129:0/1732777230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bafee0 0x5615e6bc1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eea0a640 1 -- 172.21.15.129:0/1732777230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e67e60f0 con 0x5615e6bb1a70 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96ee209640 1 --2- 172.21.15.129:0/1732777230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bafee0 0x5615e6bc1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96eea0a640 1 --2- 172.21.15.129:0/1732777230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bb1a70 0x5615e6bb4bf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f96e00ef190 tx=0x7f96e00bbe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307ec7a2e4ced329 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.403+0000 7f96d3fff640 1 -- 172.21.15.129:0/1732777230 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00bd930 con 0x5615e6bb1a70 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96d3fff640 1 -- 172.21.15.129:0/1732777230 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e00097e0 con 0x5615e6bb1a70 2024-04-26T21:20:50.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96d3fff640 1 -- 172.21.15.129:0/1732777230 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00dcaf0 con 0x5615e6bb1a70 2024-04-26T21:20:50.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96f3186740 1 -- 172.21.15.129:0/1732777230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bb1a70 msgr2=0x5615e6bb4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96f3186740 1 --2- 172.21.15.129:0/1732777230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bb1a70 0x5615e6bb4bf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f96e00ef190 tx=0x7f96e00bbe40 comp rx=0 tx=0).stop 2024-04-26T21:20:50.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96f3186740 1 -- 172.21.15.129:0/1732777230 shutdown_connections 2024-04-26T21:20:50.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96f3186740 1 --2- 172.21.15.129:0/1732777230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bb1a70 0x5615e6bb4bf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96f3186740 1 --2- 172.21.15.129:0/1732777230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb10b0 0x5615e6ba6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96f3186740 1 --2- 172.21.15.129:0/1732777230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bafee0 0x5615e6bc1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96f3186740 1 -- 172.21.15.129:0/1732777230 >> 172.21.15.129:0/1732777230 conn(0x5615e6bab3d0 msgr2=0x5615e6bab7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96f3186740 1 -- 172.21.15.129:0/1732777230 shutdown_connections 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.404+0000 7f96f3186740 1 -- 172.21.15.129:0/1732777230 wait complete. 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96f3186740 1 -- start start 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96f3186740 1 Processor -- start 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96f3186740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bc0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96f3186740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 0x5615e6bbd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96f3186740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 0x5615e6bbdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96f3186740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81aa9a0 con 0x5615e6bafee0 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96f3186740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81abe10 con 0x5615e6bb10b0 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96f3186740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81ab070 con 0x5615e6bb1a70 2024-04-26T21:20:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96f3186740 10 client.0 _ll_register_callbacks cb 0x7f96e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96ee209640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bc0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96ee209640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bc0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40870/0 (socket says 172.21.15.129:40870) 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96ee209640 1 -- 172.21.15.129:0/3409973584 learned_addr learned my addr 172.21.15.129:0/3409973584 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96eda08640 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 0x5615e6bbd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96eea0a640 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 0x5615e6bbdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96ee209640 1 -- 172.21.15.129:0/3409973584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 msgr2=0x5615e6bbdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96ee209640 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 0x5615e6bbdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96ee209640 1 -- 172.21.15.129:0/3409973584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 msgr2=0x5615e6bbd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96ee209640 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 0x5615e6bbd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96ee209640 1 -- 172.21.15.129:0/3409973584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e6ad54b0 con 0x5615e6bafee0 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96eda08640 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 0x5615e6bbd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96ee209640 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bc0d60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009400 tx=0x7f96e40052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984f47021842f45a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.405+0000 7f96d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.406+0000 7f96d1ffb640 1 -- 172.21.15.129:0/3409973584 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e402cb50 con 0x5615e6bafee0 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.406+0000 7f96f3186740 1 -- 172.21.15.129:0/3409973584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e00097e0 con 0x5615e6bafee0 2024-04-26T21:20:50.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.406+0000 7f96d1ffb640 1 -- 172.21.15.129:0/3409973584 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f96e402ccf0 con 0x5615e6bafee0 2024-04-26T21:20:50.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.406+0000 7f96d1ffb640 1 -- 172.21.15.129:0/3409973584 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f96e403c050 con 0x5615e6bafee0 2024-04-26T21:20:50.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.406+0000 7f96f3186740 10 client.15702 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:50.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.406+0000 7f96f3186740 1 -- 172.21.15.129:0/3409973584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5615e6c2b510 con 0x5615e6bafee0 2024-04-26T21:20:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96d1ffb640 1 -- 172.21.15.129:0/3409973584 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e00097e0 con 0x5615e6bafee0 2024-04-26T21:20:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96d1ffb640 1 -- 172.21.15.129:0/3409973584 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f96e4003610 con 0x5615e6bafee0 2024-04-26T21:20:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96f3186740 10 client.15702 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96f3186740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96f3186740 20 client.15702 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96f3186740 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5615e6bc5e10 0x5615e6c2dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96f3186740 4 client.15702 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:20:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96f3186740 1 -- 172.21.15.129:0/3409973584 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615e6b1c800 con 0x5615e6bc5e10 2024-04-26T21:20:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96eda08640 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5615e6bc5e10 0x5615e6c2dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96d1ffb640 10 client.15702 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:50.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.409+0000 7f96eda08640 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5615e6bc5e10 0x5615e6c2dd70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc505acf8dd238 server_cookie=730565328259d653 in_seq=0 out_seq=0 2024-04-26T21:20:50.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.416+0000 7f96d1ffb640 1 -- 172.21.15.129:0/3409973584 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5615e6b1c800 con 0x5615e6bc5e10 2024-04-26T21:20:50.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.416+0000 7f96d1ffb640 10 client.15702 handle_command_reply: tid=0 2024-04-26T21:20:50.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.416+0000 7f96f3186740 1 client.15702 shutdown 2024-04-26T21:20:50.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.416+0000 7f96f3186740 1 -- 172.21.15.129:0/3409973584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 msgr2=0x5615e6bc0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.416+0000 7f96f3186740 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bc0d60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009400 tx=0x7f96e40052c0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f3186740 1 -- 172.21.15.129:0/3409973584 shutdown_connections 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f3186740 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5615e6bc5e10 0x5615e6c2dd70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f3186740 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5615e6bb10b0 0x5615e6bbd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f3186740 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615e6bb1a70 0x5615e6bbdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96d1ffb640 0 client.15702 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f3186740 1 --2- 172.21.15.129:0/3409973584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615e6bafee0 0x5615e6bc0d60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f3186740 1 -- 172.21.15.129:0/3409973584 >> 172.21.15.129:0/3409973584 conn(0x5615e6bab3d0 msgr2=0x5615e6ba6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f3186740 1 -- 172.21.15.129:0/3409973584 shutdown_connections 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f3186740 1 -- 172.21.15.129:0/3409973584 wait complete. 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f3186740 20 client.15702 trim_cache size 0 max 16384 2024-04-26T21:20:50.419 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:20:50.420 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:20:50.420 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:20:50.420 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:20:50.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 -- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96d40a3d90 msgr2=0x7f96d40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96d40a3d90 0x7f96d40a6250 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96e819c010 tx=0x7f96d800a2d0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 -- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96e8102550 msgr2=0x7f96e81a20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96e8102550 0x7f96e81a20b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f96e002ad00 tx=0x7f96e00317b0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 -- 172.21.15.129:0/2011269754 shutdown_connections 2024-04-26T21:20:50.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f96d40a3d90 0x7f96d40a6250 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96e8101bb0 0x7f96e819b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96e8102550 0x7f96e81a20b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 --2- 172.21.15.129:0/2011269754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101250 0x7f96e819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 -- 172.21.15.129:0/2011269754 >> 172.21.15.129:0/2011269754 conn(0x7f96e80fcdf0 msgr2=0x7f96e80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 -- 172.21.15.129:0/2011269754 shutdown_connections 2024-04-26T21:20:50.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.417+0000 7f96f046d640 1 -- 172.21.15.129:0/2011269754 wait complete. 2024-04-26T21:20:50.429 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:20:50.429 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:50.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.536+0000 7f15da252640 1 -- 172.21.15.129:0/1181720951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d41033f0 msgr2=0x7f15d41057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.536+0000 7f15da252640 1 --2- 172.21.15.129:0/1181720951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d41033f0 0x7f15d41057e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009900 tx=0x7f15c402a410 comp rx=0 tx=0).stop 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.536+0000 7f15da252640 1 -- 172.21.15.129:0/1181720951 shutdown_connections 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.536+0000 7f15da252640 1 --2- 172.21.15.129:0/1181720951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15d4108650 0x7f15d410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.536+0000 7f15da252640 1 --2- 172.21.15.129:0/1181720951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d4105d20 0x7f15d4108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.536+0000 7f15da252640 1 --2- 172.21.15.129:0/1181720951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d41033f0 0x7f15d41057e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.536+0000 7f15da252640 1 -- 172.21.15.129:0/1181720951 >> 172.21.15.129:0/1181720951 conn(0x7f15d40fcde0 msgr2=0x7f15d40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.536+0000 7f15da252640 1 -- 172.21.15.129:0/1181720951 shutdown_connections 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.536+0000 7f15da252640 1 -- 172.21.15.129:0/1181720951 wait complete. 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15da252640 1 Processor -- start 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15da252640 1 -- start start 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15da252640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d41033f0 0x7f15d4102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15da252640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4105d20 0x7f15d41013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15da252640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15d4108650 0x7f15d41018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15da252640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15d410e4f0 con 0x7f15d4108650 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15da252640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15d410e370 con 0x7f15d4105d20 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15da252640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f15d410e670 con 0x7f15d41033f0 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15d2ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4105d20 0x7f15d41013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15d2ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4105d20 0x7f15d41013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:44872/0 (socket says 172.21.15.129:44872) 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15d37fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d41033f0 0x7f15d4102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15d2ffd640 1 -- 172.21.15.129:0/267422216 learned_addr learned my addr 172.21.15.129:0/267422216 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.537+0000 7f15d3fff640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15d4108650 0x7f15d41018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d2ffd640 1 -- 172.21.15.129:0/267422216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d41033f0 msgr2=0x7f15d4102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d2ffd640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d41033f0 0x7f15d4102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d2ffd640 1 -- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15d4108650 msgr2=0x7f15d41018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d2ffd640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15d4108650 0x7f15d41018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d2ffd640 1 -- 172.21.15.129:0/267422216 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d4102170 con 0x7f15d4105d20 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d37fe640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d41033f0 0x7f15d4102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d2ffd640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4105d20 0x7f15d41013a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f15c800d770 tx=0x7f15c800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d0ff9640 1 -- 172.21.15.129:0/267422216 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8017070 con 0x7f15d4105d20 2024-04-26T21:20:50.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d41a2ce0 con 0x7f15d4105d20 2024-04-26T21:20:50.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d0ff9640 1 -- 172.21.15.129:0/267422216 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15c8009c30 con 0x7f15d4105d20 2024-04-26T21:20:50.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15d0ff9640 1 -- 172.21.15.129:0/267422216 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c800f620 con 0x7f15d4105d20 2024-04-26T21:20:50.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.538+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41a31d0 con 0x7f15d4105d20 2024-04-26T21:20:50.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.540+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d40fe8d0 con 0x7f15d4105d20 2024-04-26T21:20:50.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.541+0000 7f15d0ff9640 1 -- 172.21.15.129:0/267422216 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f15c801a070 con 0x7f15d4105d20 2024-04-26T21:20:50.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.542+0000 7f15d0ff9640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15bc0a3c40 0x7f15bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.542+0000 7f15d0ff9640 1 -- 172.21.15.129:0/267422216 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f15c80d0030 con 0x7f15d4105d20 2024-04-26T21:20:50.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.542+0000 7f15d37fe640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15bc0a3c40 0x7f15bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.543+0000 7f15d37fe640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15bc0a3c40 0x7f15bc0a6100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f15c4031500 tx=0x7f15c4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.548+0000 7f15d0ff9640 1 -- 172.21.15.129:0/267422216 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f15c8009de0 con 0x7f15d4105d20 2024-04-26T21:20:50.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:50 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:20:50.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:50 smithi129 ceph-mon[29237]: pgmap v435: 97 pgs: 97 active+clean; 292 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 735 op/s 2024-04-26T21:20:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:50 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:20:50.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:50 smithi171 ceph-mon[33478]: pgmap v435: 97 pgs: 97 active+clean; 292 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 735 op/s 2024-04-26T21:20:50.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.839+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15d402dc80 con 0x7f15d4105d20 2024-04-26T21:20:50.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.840+0000 7f15d0ff9640 1 -- 172.21.15.129:0/267422216 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f15c809c250 con 0x7f15d4105d20 2024-04-26T21:20:50.845 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:20:50.846 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:20:50.846 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:20:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.864+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15bc0a3c40 msgr2=0x7f15bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.864+0000 7f15da252640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15bc0a3c40 0x7f15bc0a6100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f15c4031500 tx=0x7f15c4005a20 comp rx=0 tx=0).stop 2024-04-26T21:20:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.864+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4105d20 msgr2=0x7f15d41013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.864+0000 7f15da252640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4105d20 0x7f15d41013a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f15c800d770 tx=0x7f15c800dc30 comp rx=0 tx=0).stop 2024-04-26T21:20:50.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.866+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 shutdown_connections 2024-04-26T21:20:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.866+0000 7f15da252640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f15bc0a3c40 0x7f15bc0a6100 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.866+0000 7f15da252640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4105d20 0x7f15d41013a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.866+0000 7f15da252640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15d4108650 0x7f15d41018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.866+0000 7f15da252640 1 --2- 172.21.15.129:0/267422216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d41033f0 0x7f15d4102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.866+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 >> 172.21.15.129:0/267422216 conn(0x7f15d40fcde0 msgr2=0x7f15d4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.866+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 shutdown_connections 2024-04-26T21:20:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:50.867+0000 7f15da252640 1 -- 172.21.15.129:0/267422216 wait complete. 2024-04-26T21:20:50.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:50 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:20:50.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:50 smithi150 ceph-mon[32290]: pgmap v435: 97 pgs: 97 active+clean; 292 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 735 op/s 2024-04-26T21:20:50.893 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:20:51.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.073+0000 7f54b64b0640 1 -- 172.21.15.129:0/1487496303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0108660 msgr2=0x7f54b010aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.073+0000 7f54b64b0640 1 --2- 172.21.15.129:0/1487496303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0108660 0x7f54b010aab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009ca0 tx=0x7f54a802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.073+0000 7f54b64b0640 1 -- 172.21.15.129:0/1487496303 shutdown_connections 2024-04-26T21:20:51.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.073+0000 7f54b64b0640 1 --2- 172.21.15.129:0/1487496303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0108660 0x7f54b010aab0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.073+0000 7f54b64b0640 1 --2- 172.21.15.129:0/1487496303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54b0105d30 0x7f54b0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.073+0000 7f54b64b0640 1 --2- 172.21.15.129:0/1487496303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54b0103400 0x7f54b01057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.073+0000 7f54b64b0640 1 -- 172.21.15.129:0/1487496303 >> 172.21.15.129:0/1487496303 conn(0x7f54b00fcdf0 msgr2=0x7f54b00ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.073+0000 7f54b64b0640 1 -- 172.21.15.129:0/1487496303 shutdown_connections 2024-04-26T21:20:51.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.073+0000 7f54b64b0640 1 -- 172.21.15.129:0/1487496303 wait complete. 2024-04-26T21:20:51.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54b64b0640 1 Processor -- start 2024-04-26T21:20:51.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54b64b0640 1 -- start start 2024-04-26T21:20:51.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54b64b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0103400 0x7f54b0102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54b64b0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54b0105d30 0x7f54b01013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54b64b0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54b0108660 0x7f54b01018f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54b64b0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54b010e4d0 con 0x7f54b0105d30 2024-04-26T21:20:51.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54b64b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54b010e350 con 0x7f54b0103400 2024-04-26T21:20:51.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54b64b0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f54b010e650 con 0x7f54b0108660 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54affff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0103400 0x7f54b0102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54b4a4d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54b0108660 0x7f54b01018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54b0105d30 0x7f54b01013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54affff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0103400 0x7f54b0102d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:44894/0 (socket says 172.21.15.129:44894) 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.074+0000 7f54affff640 1 -- 172.21.15.129:0/39978509 learned_addr learned my addr 172.21.15.129:0/39978509 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54affff640 1 -- 172.21.15.129:0/39978509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54b0108660 msgr2=0x7f54b01018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54affff640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54b0108660 0x7f54b01018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54affff640 1 -- 172.21.15.129:0/39978509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54b0105d30 msgr2=0x7f54b01013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54affff640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54b0105d30 0x7f54b01013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54affff640 1 -- 172.21.15.129:0/39978509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b0102180 con 0x7f54b0103400 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54affff640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0103400 0x7f54b0102d70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e9f0 tx=0x7f54a000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54ad7fa640 1 -- 172.21.15.129:0/39978509 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a000cd80 con 0x7f54b0103400 2024-04-26T21:20:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54ad7fa640 1 -- 172.21.15.129:0/39978509 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a00054d0 con 0x7f54b0103400 2024-04-26T21:20:51.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54b64b0640 1 -- 172.21.15.129:0/39978509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b01a2cc0 con 0x7f54b0103400 2024-04-26T21:20:51.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54ad7fa640 1 -- 172.21.15.129:0/39978509 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0010690 con 0x7f54b0103400 2024-04-26T21:20:51.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.075+0000 7f54b64b0640 1 -- 172.21.15.129:0/39978509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b01a31c0 con 0x7f54b0103400 2024-04-26T21:20:51.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.078+0000 7f54b91c9740 1 Processor -- start 2024-04-26T21:20:51.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.078+0000 7f54b91c9740 1 -- start start 2024-04-26T21:20:51.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.078+0000 7f54ad7fa640 1 -- 172.21.15.129:0/39978509 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f54a0010830 con 0x7f54b0103400 2024-04-26T21:20:51.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x5561529f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f79f0 0x5561529f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f83b0 0x5561529fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b91c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55615295ad80 con 0x5561529f6820 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b91c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556152914a20 con 0x5561529f83b0 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b91c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556152956d40 con 0x5561529f79f0 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b4a4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x5561529f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b4a4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x5561529f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40904/0 (socket says 172.21.15.129:40904) 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b4a4d640 1 -- 172.21.15.129:0/2994408728 learned_addr learned my addr 172.21.15.129:0/2994408728 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54af7fe640 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f83b0 0x5561529fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54affff640 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f79f0 0x5561529f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54af7fe640 1 -- 172.21.15.129:0/2994408728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f79f0 msgr2=0x5561529f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54af7fe640 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f79f0 0x5561529f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54af7fe640 1 -- 172.21.15.129:0/2994408728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 msgr2=0x5561529f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54af7fe640 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x5561529f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54ad7fa640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f54980a3cc0 0x7f54980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54af7fe640 1 -- 172.21.15.129:0/2994408728 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55615291fc30 con 0x5561529f83b0 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b4a4d640 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x5561529f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54b4a4d640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f54980a3cc0 0x7f54980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54ad7fa640 1 -- 172.21.15.129:0/39978509 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f54a00d0930 con 0x7f54b0103400 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.079+0000 7f54af7fe640 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f83b0 0x5561529fec40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54b006ba30 tx=0x7f54a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75643fab3b90eabc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.080+0000 7f54967fc640 1 -- 172.21.15.129:0/2994408728 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a401e070 con 0x5561529f83b0 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.080+0000 7f54967fc640 1 -- 172.21.15.129:0/2994408728 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a4009d70 con 0x5561529f83b0 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.080+0000 7f54967fc640 1 -- 172.21.15.129:0/2994408728 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4018730 con 0x5561529f83b0 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.080+0000 7f54b91c9740 1 -- 172.21.15.129:0/2994408728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f83b0 msgr2=0x5561529fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.080+0000 7f54b4a4d640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f54980a3cc0 0x7f54980a6180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009430 tx=0x7f54a8009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.080+0000 7f54b91c9740 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f83b0 0x5561529fec40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54b006ba30 tx=0x7f54a4011810 comp rx=0 tx=0).stop 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 -- 172.21.15.129:0/2994408728 shutdown_connections 2024-04-26T21:20:51.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f83b0 0x5561529fec40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f79f0 0x5561529f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 --2- 172.21.15.129:0/2994408728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x5561529f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 -- 172.21.15.129:0/2994408728 >> 172.21.15.129:0/2994408728 conn(0x556152937910 msgr2=0x5561529f2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 -- 172.21.15.129:0/2994408728 shutdown_connections 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 -- 172.21.15.129:0/2994408728 wait complete. 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 -- start start 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 Processor -- start 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x556152a00ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a01000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.081+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 0x556152a08040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54affff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x556152a00ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a01000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54b91c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5561529ff3c0 con 0x5561529f6820 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54affff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x556152a00ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40910/0 (socket says 172.21.15.129:40910) 2024-04-26T21:20:51.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54b91c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5561528e1dc0 con 0x5561529f79f0 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a01000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:44914/0 (socket says 172.21.15.129:44914) 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54affff640 1 -- 172.21.15.129:0/3667771261 learned_addr learned my addr 172.21.15.129:0/3667771261 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54b91c9740 1 -- 172.21.15.129:0/3667771261 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5561529ff540 con 0x5561529f83b0 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54b4a4d640 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 0x556152a08040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54b91c9740 10 client.0 _ll_register_callbacks cb 0x5561528fd600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54af7fe640 1 -- 172.21.15.129:0/3667771261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 msgr2=0x556152a08040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54af7fe640 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 0x556152a08040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54af7fe640 1 -- 172.21.15.129:0/3667771261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 msgr2=0x556152a00ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54af7fe640 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x556152a00ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54af7fe640 1 -- 172.21.15.129:0/3667771261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55615266c3f0 con 0x5561529f79f0 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54af7fe640 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a01000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5561529f2fd0 tx=0x7f54a40056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45423302e5788a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f548f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f548f7fe640 1 -- 172.21.15.129:0/3667771261 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a401e070 con 0x5561529f79f0 2024-04-26T21:20:51.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f54b91c9740 1 -- 172.21.15.129:0/3667771261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a4009d70 con 0x5561529f79f0 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.082+0000 7f548f7fe640 1 -- 172.21.15.129:0/3667771261 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f54a4003800 con 0x5561529f79f0 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f548f7fe640 1 -- 172.21.15.129:0/3667771261 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f54a4003a00 con 0x5561529f79f0 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f54b91c9740 10 client.25325 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f54b91c9740 1 -- 172.21.15.129:0/3667771261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5561527367d0 con 0x5561529f79f0 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f548f7fe640 1 -- 172.21.15.129:0/3667771261 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a4009d70 con 0x5561529f79f0 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f548f7fe640 1 -- 172.21.15.129:0/3667771261 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f54a40184b0 con 0x5561529f79f0 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f54b91c9740 10 client.25325 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f54b91c9740 10 client.25325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f54b91c9740 20 client.25325 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556152a0c690 0x556152a0eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f54b91c9740 4 client.25325 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f54b91c9740 1 -- 172.21.15.129:0/3667771261 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556152968990 con 0x556152a0c690 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f54affff640 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556152a0c690 0x556152a0eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.083+0000 7f548f7fe640 10 client.25325 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:51.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.084+0000 7f54affff640 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556152a0c690 0x556152a0eaf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ffa705785e7155 server_cookie=b482d2a09f9bca3e in_seq=0 out_seq=0 2024-04-26T21:20:51.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f548f7fe640 1 -- 172.21.15.129:0/3667771261 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556152968990 con 0x556152a0c690 2024-04-26T21:20:51.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f548f7fe640 10 client.25325 handle_command_reply: tid=0 2024-04-26T21:20:51.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 client.25325 shutdown 2024-04-26T21:20:51.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 -- 172.21.15.129:0/3667771261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 msgr2=0x556152a01000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a01000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5561529f2fd0 tx=0x7f54a40056c0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 -- 172.21.15.129:0/3667771261 shutdown_connections 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556152a0c690 0x556152a0eaf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a01000 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 0x556152a08040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3667771261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f6820 0x556152a00ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 -- 172.21.15.129:0/3667771261 >> 172.21.15.129:0/3667771261 conn(0x556152937910 msgr2=0x5561529f1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f548f7fe640 0 client.25325 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 -- 172.21.15.129:0/3667771261 shutdown_connections 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.086+0000 7f54b91c9740 1 -- 172.21.15.129:0/3667771261 wait complete. 2024-04-26T21:20:51.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.087+0000 7f54b91c9740 20 client.25325 trim_cache size 0 max 16384 2024-04-26T21:20:51.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.100+0000 7f54b91c9740 1 Processor -- start 2024-04-26T21:20:51.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.100+0000 7f54b91c9740 1 -- start start 2024-04-26T21:20:51.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.100+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f6820 0x5561529f5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.100+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a08000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.100+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f83b0 0x5561529f5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.100+0000 7f54b91c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01aa000 con 0x5561529f83b0 2024-04-26T21:20:51.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.100+0000 7f54b91c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01ac880 con 0x5561529f79f0 2024-04-26T21:20:51.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.100+0000 7f54b91c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01aa670 con 0x5561529f6820 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54b4a4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f83b0 0x5561529f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a08000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54b4a4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f83b0 0x5561529f5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40922/0 (socket says 172.21.15.129:40922) 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a08000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:44916/0 (socket says 172.21.15.129:44916) 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54b4a4d640 1 -- 172.21.15.129:0/298011334 learned_addr learned my addr 172.21.15.129:0/298011334 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54affff640 1 --2- 172.21.15.129:0/298011334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f6820 0x5561529f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54af7fe640 1 -- 172.21.15.129:0/298011334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f6820 msgr2=0x5561529f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54af7fe640 1 --2- 172.21.15.129:0/298011334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f6820 0x5561529f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54af7fe640 1 -- 172.21.15.129:0/298011334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f83b0 msgr2=0x5561529f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54af7fe640 1 --2- 172.21.15.129:0/298011334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f83b0 0x5561529f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54af7fe640 1 -- 172.21.15.129:0/298011334 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561529f2370 con 0x5561529f79f0 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f54af7fe640 1 --2- 172.21.15.129:0/298011334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a08000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54a4019000 tx=0x7f54a4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df00b9d0d54f1ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f5495ffb640 1 -- 172.21.15.129:0/298011334 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a401e040 con 0x5561529f79f0 2024-04-26T21:20:51.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f5495ffb640 1 -- 172.21.15.129:0/298011334 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a4028070 con 0x5561529f79f0 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.101+0000 7f5495ffb640 1 -- 172.21.15.129:0/298011334 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a40236a0 con 0x5561529f79f0 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.102+0000 7f54b91c9740 1 -- 172.21.15.129:0/298011334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 msgr2=0x556152a08000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.102+0000 7f54b91c9740 1 --2- 172.21.15.129:0/298011334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a08000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54a4019000 tx=0x7f54a4005f90 comp rx=0 tx=0).stop 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.102+0000 7f54b91c9740 1 -- 172.21.15.129:0/298011334 shutdown_connections 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.102+0000 7f54b91c9740 1 --2- 172.21.15.129:0/298011334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f79f0 0x556152a08000 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.102+0000 7f54b91c9740 1 --2- 172.21.15.129:0/298011334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f83b0 0x5561529f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.102+0000 7f54b91c9740 1 --2- 172.21.15.129:0/298011334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f6820 0x5561529f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.102+0000 7f54b91c9740 1 -- 172.21.15.129:0/298011334 >> 172.21.15.129:0/298011334 conn(0x556152937910 msgr2=0x5561529fb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.102+0000 7f54b91c9740 1 -- 172.21.15.129:0/298011334 shutdown_connections 2024-04-26T21:20:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.102+0000 7f54b91c9740 1 -- 172.21.15.129:0/298011334 wait complete. 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b91c9740 1 -- start start 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b91c9740 1 Processor -- start 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f6820 0x556152a050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f79f0 0x556152a05620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b91c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 0x556152a06630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b91c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01aad40 con 0x5561529f79f0 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b91c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01ac1b0 con 0x5561529f6820 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b91c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01ab410 con 0x5561529f83b0 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b91c9740 10 client.0 _ll_register_callbacks cb 0x7f54a401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f79f0 0x556152a05620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f79f0 0x556152a05620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40928/0 (socket says 172.21.15.129:40928) 2024-04-26T21:20:51.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54af7fe640 1 -- 172.21.15.129:0/3435819275 learned_addr learned my addr 172.21.15.129:0/3435819275 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54affff640 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f6820 0x556152a050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54b4a4d640 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 0x556152a06630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54af7fe640 1 -- 172.21.15.129:0/3435819275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 msgr2=0x556152a06630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54af7fe640 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 0x556152a06630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54af7fe640 1 -- 172.21.15.129:0/3435819275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f6820 msgr2=0x556152a050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.103+0000 7f54af7fe640 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f6820 0x556152a050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f54af7fe640 1 -- 172.21.15.129:0/3435819275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561529ed8b0 con 0x5561529f79f0 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f54af7fe640 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f79f0 0x556152a05620 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f54a40188f0 tx=0x7f54a4005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa9850554145c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f548f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f548f7fe640 1 -- 172.21.15.129:0/3435819275 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a403b020 con 0x5561529f79f0 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f548f7fe640 1 -- 172.21.15.129:0/3435819275 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f54a40183b0 con 0x5561529f79f0 2024-04-26T21:20:51.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f54b91c9740 1 -- 172.21.15.129:0/3435819275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a4028070 con 0x5561529f79f0 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f548f7fe640 1 -- 172.21.15.129:0/3435819275 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f54a4046050 con 0x5561529f79f0 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f54b91c9740 10 client.15726 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f54b91c9740 1 -- 172.21.15.129:0/3435819275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556152a07d50 con 0x5561529f79f0 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f548f7fe640 1 -- 172.21.15.129:0/3435819275 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a4028070 con 0x5561529f79f0 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.104+0000 7f548f7fe640 1 -- 172.21.15.129:0/3435819275 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f54a4023bd0 con 0x5561529f79f0 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.105+0000 7f54b91c9740 10 client.15726 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.105+0000 7f54b91c9740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.105+0000 7f54b91c9740 20 client.15726 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.105+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556152a72d80 0x556152a751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.105+0000 7f54b91c9740 4 client.15726 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.105+0000 7f54b91c9740 1 -- 172.21.15.129:0/3435819275 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556152962800 con 0x556152a72d80 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.105+0000 7f54affff640 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556152a72d80 0x556152a751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.105+0000 7f548f7fe640 10 client.15726 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.105+0000 7f54affff640 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556152a72d80 0x556152a751a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322c084308f7706 server_cookie=3467b836393fdc4f in_seq=0 out_seq=0 2024-04-26T21:20:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f548f7fe640 1 -- 172.21.15.129:0/3435819275 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556152962800 con 0x556152a72d80 2024-04-26T21:20:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f548f7fe640 10 client.15726 handle_command_reply: tid=0 2024-04-26T21:20:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 client.15726 shutdown 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 -- 172.21.15.129:0/3435819275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f79f0 msgr2=0x556152a05620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f79f0 0x556152a05620 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f54a40188f0 tx=0x7f54a4005370 comp rx=0 tx=0).stop 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 -- 172.21.15.129:0/3435819275 shutdown_connections 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x556152a72d80 0x556152a751a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5561529f83b0 0x556152a06630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561529f79f0 0x556152a05620 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f548f7fe640 0 client.15726 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 --2- 172.21.15.129:0/3435819275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561529f6820 0x556152a050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 -- 172.21.15.129:0/3435819275 >> 172.21.15.129:0/3435819275 conn(0x556152937910 msgr2=0x5561529f09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.110+0000 7f54b91c9740 1 -- 172.21.15.129:0/3435819275 shutdown_connections 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b91c9740 1 -- 172.21.15.129:0/3435819275 wait complete. 2024-04-26T21:20:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b91c9740 20 client.15726 trim_cache size 0 max 16384 2024-04-26T21:20:51.116 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:20:51.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 -- 172.21.15.129:0/39978509 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f54980a3cc0 msgr2=0x7f54980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f54980a3cc0 0x7f54980a6180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009430 tx=0x7f54a8009360 comp rx=0 tx=0).stop 2024-04-26T21:20:51.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 -- 172.21.15.129:0/39978509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0103400 msgr2=0x7f54b0102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0103400 0x7f54b0102d70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e9f0 tx=0x7f54a000edf0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 -- 172.21.15.129:0/39978509 shutdown_connections 2024-04-26T21:20:51.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f54980a3cc0 0x7f54980a6180 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54b0108660 0x7f54b01018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54b0105d30 0x7f54b01013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 --2- 172.21.15.129:0/39978509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54b0103400 0x7f54b0102d70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 -- 172.21.15.129:0/39978509 >> 172.21.15.129:0/39978509 conn(0x7f54b00fcdf0 msgr2=0x7f54b0103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 -- 172.21.15.129:0/39978509 shutdown_connections 2024-04-26T21:20:51.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:51.111+0000 7f54b64b0640 1 -- 172.21.15.129:0/39978509 wait complete. 2024-04-26T21:20:51.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:51 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/267422216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:20:51.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:51 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/267422216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:20:51.768 INFO:tasks.workunit.client.0.smithi129.stdout: 62 1368044 327 969960 321 695744 1280 2024-04-26T21:20:51.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:51 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/267422216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:20:52.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:20:52.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:52.132 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:20:52.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:52 smithi129 ceph-mon[29237]: pgmap v436: 97 pgs: 97 active+clean; 290 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.3 MiB/s wr, 728 op/s 2024-04-26T21:20:52.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:52 smithi171 ceph-mon[33478]: pgmap v436: 97 pgs: 97 active+clean; 290 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.3 MiB/s wr, 728 op/s 2024-04-26T21:20:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.778+0000 7fd5fd567640 1 -- 172.21.15.129:0/3669176853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f810a2f0 msgr2=0x7fd5f8111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.778+0000 7fd5fd567640 1 --2- 172.21.15.129:0/3669176853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f810a2f0 0x7fd5f8111b90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800aa10 tx=0x7fd5e802b750 comp rx=0 tx=0).stop 2024-04-26T21:20:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.779+0000 7fd5fd567640 1 -- 172.21.15.129:0/3669176853 shutdown_connections 2024-04-26T21:20:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.779+0000 7fd5fd567640 1 --2- 172.21.15.129:0/3669176853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f8109930 0x7fd5f8109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.779+0000 7fd5fd567640 1 --2- 172.21.15.129:0/3669176853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f810a2f0 0x7fd5f8111b90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.779+0000 7fd5fd567640 1 --2- 172.21.15.129:0/3669176853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f810ad70 0x7fd5f810b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.779+0000 7fd5fd567640 1 -- 172.21.15.129:0/3669176853 >> 172.21.15.129:0/3669176853 conn(0x7fd5f806f840 msgr2=0x7fd5f8071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:52.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.784+0000 7fd5fd567640 1 -- 172.21.15.129:0/3669176853 shutdown_connections 2024-04-26T21:20:52.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.784+0000 7fd5fd567640 1 -- 172.21.15.129:0/3669176853 wait complete. 2024-04-26T21:20:52.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5fd567640 1 Processor -- start 2024-04-26T21:20:52.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5fd567640 1 -- start start 2024-04-26T21:20:52.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5fd567640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f8109930 0x7fd5f810d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5fd567640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f810a2f0 0x7fd5f810bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5fd567640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f810ad70 0x7fd5f810c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5fd567640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8114a20 con 0x7fd5f810ad70 2024-04-26T21:20:52.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5fd567640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81148a0 con 0x7fd5f8109930 2024-04-26T21:20:52.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5fd567640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8114ba0 con 0x7fd5f810a2f0 2024-04-26T21:20:52.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f8109930 0x7fd5f810d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f8109930 0x7fd5f810d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60552/0 (socket says 172.21.15.129:60552) 2024-04-26T21:20:52.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.785+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/1863034918 learned_addr learned my addr 172.21.15.129:0/1863034918 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:52.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.786+0000 7fd5f77fe640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f810ad70 0x7fd5f810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.786+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f810a2f0 0x7fd5f810bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.786+0000 7fd5f77fe640 1 -- 172.21.15.129:0/1863034918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f810a2f0 msgr2=0x7fd5f810bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:52.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.786+0000 7fd5f77fe640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f810a2f0 0x7fd5f810bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.786+0000 7fd5f77fe640 1 -- 172.21.15.129:0/1863034918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f8109930 msgr2=0x7fd5f810d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:52.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.786+0000 7fd5f77fe640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f8109930 0x7fd5f810d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.786+0000 7fd5f77fe640 1 -- 172.21.15.129:0/1863034918 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f810c7a0 con 0x7fd5f810ad70 2024-04-26T21:20:52.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.787+0000 7fd5f77fe640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f810ad70 0x7fd5f810c260 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800a340 tx=0x7fd5e8030c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:52.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.797+0000 7fd5dffff640 1 -- 172.21.15.129:0/1863034918 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e803f070 con 0x7fd5f810ad70 2024-04-26T21:20:52.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.798+0000 7fd5dffff640 1 -- 172.21.15.129:0/1863034918 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e8003eb0 con 0x7fd5f810ad70 2024-04-26T21:20:52.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.798+0000 7fd5dffff640 1 -- 172.21.15.129:0/1863034918 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8036040 con 0x7fd5f810ad70 2024-04-26T21:20:52.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.798+0000 7fd5fd567640 1 -- 172.21.15.129:0/1863034918 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f810ca30 con 0x7fd5f810ad70 2024-04-26T21:20:52.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.798+0000 7fd5fd567640 1 -- 172.21.15.129:0/1863034918 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f81afd00 con 0x7fd5f810ad70 2024-04-26T21:20:52.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.806+0000 7fd60027d740 1 Processor -- start 2024-04-26T21:20:52.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.806+0000 7fd60027d740 1 -- start start 2024-04-26T21:20:52.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.806+0000 7fd60027d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefb7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.806+0000 7fd60027d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefbcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd60027d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefbfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd60027d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5637aef1cb00 con 0x5637aefb7a80 2024-04-26T21:20:52.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd60027d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5637aeeb8920 con 0x5637aefbd5e0 2024-04-26T21:20:52.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd60027d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5637aef18ac0 con 0x5637aefb8470 2024-04-26T21:20:52.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd5f67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefb7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefbcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefbcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:55206/0 (socket says 172.21.15.129:55206) 2024-04-26T21:20:52.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/111014743 learned_addr learned my addr 172.21.15.129:0/111014743 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:52.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/111014743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 msgr2=0x5637aefbfa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:20:52.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/111014743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefbfa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/111014743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 msgr2=0x5637aefb7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:52.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.807+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/111014743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefb7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.814+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/111014743 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637aecf67e0 con 0x5637aefb8470 2024-04-26T21:20:52.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.814+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/111014743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefb7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:52.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.815+0000 7fd5dffff640 1 -- 172.21.15.129:0/1863034918 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd5e8042070 con 0x7fd5f810ad70 2024-04-26T21:20:52.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.815+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/111014743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefbcf70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00adb0 tx=0x7fd5ec00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb9625ff8edc8e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:52.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.815+0000 7fd5dd7fa640 1 -- 172.21.15.129:0/111014743 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec01d070 con 0x5637aefb8470 2024-04-26T21:20:52.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.815+0000 7fd5dd7fa640 1 -- 172.21.15.129:0/111014743 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ec010cf0 con 0x5637aefb8470 2024-04-26T21:20:52.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.816+0000 7fd5dd7fa640 1 -- 172.21.15.129:0/111014743 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec018c40 con 0x5637aefb8470 2024-04-26T21:20:52.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.817+0000 7fd60027d740 1 -- 172.21.15.129:0/111014743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 msgr2=0x5637aefbcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:52.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.817+0000 7fd60027d740 1 --2- 172.21.15.129:0/111014743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefbcf70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00adb0 tx=0x7fd5ec00f790 comp rx=0 tx=0).stop 2024-04-26T21:20:52.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.817+0000 7fd60027d740 1 -- 172.21.15.129:0/111014743 shutdown_connections 2024-04-26T21:20:52.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.817+0000 7fd60027d740 1 --2- 172.21.15.129:0/111014743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefbfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.817+0000 7fd60027d740 1 --2- 172.21.15.129:0/111014743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefbcf70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.817+0000 7fd60027d740 1 --2- 172.21.15.129:0/111014743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefb7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.817+0000 7fd60027d740 1 -- 172.21.15.129:0/111014743 >> 172.21.15.129:0/111014743 conn(0x5637aefb30f0 msgr2=0x5637aefb34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:52.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.820+0000 7fd60027d740 1 -- 172.21.15.129:0/111014743 shutdown_connections 2024-04-26T21:20:52.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.820+0000 7fd60027d740 1 -- 172.21.15.129:0/111014743 wait complete. 2024-04-26T21:20:52.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.820+0000 7fd60027d740 1 -- start start 2024-04-26T21:20:52.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.820+0000 7fd60027d740 1 Processor -- start 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd60027d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd60027d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefc24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd60027d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefc9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd60027d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5637aefc05e0 con 0x5637aefb7a80 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd60027d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5637aeeb65d0 con 0x5637aefbd5e0 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd60027d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5637aefc0760 con 0x5637aefb8470 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd60027d740 10 client.0 _ll_register_callbacks cb 0x5637aeea4080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd5f67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd5f67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59854/0 (socket says 172.21.15.129:59854) 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd5f67fc640 1 -- 172.21.15.129:0/3978140255 learned_addr learned my addr 172.21.15.129:0/3978140255 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefc24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.821+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/3978140255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 msgr2=0x5637aefc9530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:20:52.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.822+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefc9530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.822+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/3978140255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 msgr2=0x5637aefc1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:52.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.822+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.822+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/3978140255 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637aed56f70 con 0x5637aefb8470 2024-04-26T21:20:52.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.822+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:52.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.822+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefc24f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5637aefb3fc0 tx=0x7fd5ec018440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e1d7bad4ad0efca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:52.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.826+0000 7fd5dffff640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5e00a3d00 0x7fd5e00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.827+0000 7fd5dffff640 1 -- 172.21.15.129:0/1863034918 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd5e80f1c30 con 0x7fd5f810ad70 2024-04-26T21:20:52.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.827+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5e00a3d00 0x7fd5e00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.828+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5e00a3d00 0x7fd5e00a61c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000b940 tx=0x7fd5f000e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:52.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.829+0000 7fd5daff5640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-26T21:20:52.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.829+0000 7fd5daff5640 1 -- 172.21.15.129:0/3978140255 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec026020 con 0x5637aefb8470 2024-04-26T21:20:52.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.829+0000 7fd60027d740 1 -- 172.21.15.129:0/3978140255 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ec010cf0 con 0x5637aefb8470 2024-04-26T21:20:52.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.829+0000 7fd5daff5640 1 -- 172.21.15.129:0/3978140255 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd5ec010640 con 0x5637aefb8470 2024-04-26T21:20:52.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.840+0000 7fd5daff5640 1 -- 172.21.15.129:0/3978140255 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec031050 con 0x5637aefb8470 2024-04-26T21:20:52.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.840+0000 7fd60027d740 10 client.25267 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:52.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.840+0000 7fd60027d740 1 -- 172.21.15.129:0/3978140255 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5637aebee170 con 0x5637aefb8470 2024-04-26T21:20:52.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.841+0000 7fd5daff5640 1 -- 172.21.15.129:0/3978140255 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ec010cf0 con 0x5637aefb8470 2024-04-26T21:20:52.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.841+0000 7fd5daff5640 1 -- 172.21.15.129:0/3978140255 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fd5ec0046c0 con 0x5637aefb8470 2024-04-26T21:20:52.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.841+0000 7fd60027d740 10 client.25267 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:52.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.841+0000 7fd60027d740 10 client.25267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:52.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.841+0000 7fd60027d740 20 client.25267 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:52.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.841+0000 7fd60027d740 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5637aefcdd90 0x5637aefd01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.841+0000 7fd60027d740 4 client.25267 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:52.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.841+0000 7fd60027d740 1 -- 172.21.15.129:0/3978140255 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637aef2a710 con 0x5637aefcdd90 2024-04-26T21:20:52.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.841+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5637aefcdd90 0x5637aefd01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.842+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5637aefcdd90 0x5637aefd01b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a9f28d9a3fd375 server_cookie=42f5ee40b4032b6d in_seq=0 out_seq=0 2024-04-26T21:20:52.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.842+0000 7fd5daff5640 10 client.25267 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:52.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.845+0000 7fd5daff5640 1 -- 172.21.15.129:0/3978140255 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637aef2a710 con 0x5637aefcdd90 2024-04-26T21:20:52.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.845+0000 7fd5daff5640 10 client.25267 handle_command_reply: tid=0 2024-04-26T21:20:52.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd60027d740 1 client.25267 shutdown 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd60027d740 1 -- 172.21.15.129:0/3978140255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 msgr2=0x5637aefc24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd60027d740 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefc24f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5637aefb3fc0 tx=0x7fd5ec018440 comp rx=0 tx=0).stop 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd60027d740 1 -- 172.21.15.129:0/3978140255 shutdown_connections 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd60027d740 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5637aefcdd90 0x5637aefd01b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd60027d740 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefc9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd60027d740 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb8470 0x5637aefc24f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd60027d740 1 --2- 172.21.15.129:0/3978140255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd60027d740 1 -- 172.21.15.129:0/3978140255 >> 172.21.15.129:0/3978140255 conn(0x5637aefb30f0 msgr2=0x5637aefb22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.846+0000 7fd5daff5640 0 client.25267 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:52.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.847+0000 7fd60027d740 1 -- 172.21.15.129:0/3978140255 shutdown_connections 2024-04-26T21:20:52.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.847+0000 7fd60027d740 1 -- 172.21.15.129:0/3978140255 wait complete. 2024-04-26T21:20:52.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.860+0000 7fd60027d740 20 client.25267 trim_cache size 0 max 16384 2024-04-26T21:20:52.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:52 smithi150 ceph-mon[32290]: pgmap v436: 97 pgs: 97 active+clean; 290 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.3 MiB/s wr, 728 op/s 2024-04-26T21:20:52.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.920+0000 7fd60027d740 1 Processor -- start 2024-04-26T21:20:52.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.921+0000 7fd60027d740 1 -- start start 2024-04-26T21:20:52.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.921+0000 7fd60027d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd60027d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefbd5e0 0x5637aefc94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd60027d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefb6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd60027d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e400c5a0 con 0x5637aefc24f0 2024-04-26T21:20:52.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd60027d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e400ee20 con 0x5637aefb7a80 2024-04-26T21:20:52.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd60027d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e400cc10 con 0x5637aefbd5e0 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60554/0 (socket says 172.21.15.129:60554) 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/1776117993 learned_addr learned my addr 172.21.15.129:0/1776117993 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefbd5e0 0x5637aefc94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f77fe640 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefb6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/1776117993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefbd5e0 msgr2=0x5637aefc94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefbd5e0 0x5637aefc94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/1776117993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 msgr2=0x5637aefb6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefb6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f6ffd640 1 -- 172.21.15.129:0/1776117993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637aefb3dc0 con 0x5637aefb7a80 2024-04-26T21:20:52.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.922+0000 7fd5f77fe640 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefb6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:52.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.923+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00a670 tx=0x7fd5ec00fd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f149164e25b1481 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:52.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.930+0000 7fd5dcff9640 1 -- 172.21.15.129:0/1776117993 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec026070 con 0x5637aefb7a80 2024-04-26T21:20:52.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.931+0000 7fd5dcff9640 1 -- 172.21.15.129:0/1776117993 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ec00a7e0 con 0x5637aefb7a80 2024-04-26T21:20:52.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.931+0000 7fd5dcff9640 1 -- 172.21.15.129:0/1776117993 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec01d040 con 0x5637aefb7a80 2024-04-26T21:20:52.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.932+0000 7fd60027d740 1 -- 172.21.15.129:0/1776117993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefb7a80 msgr2=0x5637aefc1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:52.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.932+0000 7fd60027d740 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00a670 tx=0x7fd5ec00fd10 comp rx=0 tx=0).stop 2024-04-26T21:20:52.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.932+0000 7fd60027d740 1 -- 172.21.15.129:0/1776117993 shutdown_connections 2024-04-26T21:20:52.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.932+0000 7fd60027d740 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefb6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.932+0000 7fd60027d740 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefbd5e0 0x5637aefc94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.932+0000 7fd60027d740 1 --2- 172.21.15.129:0/1776117993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefb7a80 0x5637aefc1fb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.932+0000 7fd60027d740 1 -- 172.21.15.129:0/1776117993 >> 172.21.15.129:0/1776117993 conn(0x5637aefb30f0 msgr2=0x5637aefb98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:52.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.963+0000 7fd60027d740 1 -- 172.21.15.129:0/1776117993 shutdown_connections 2024-04-26T21:20:52.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.963+0000 7fd60027d740 1 -- 172.21.15.129:0/1776117993 wait complete. 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.963+0000 7fd60027d740 1 -- start start 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.963+0000 7fd60027d740 1 Processor -- start 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd60027d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb7a80 0x5637aefc6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd60027d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefc6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd60027d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefc7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd60027d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e400d2e0 con 0x5637aefc24f0 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd60027d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e400e750 con 0x5637aefbd5e0 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd60027d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e400d9b0 con 0x5637aefb7a80 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd60027d740 10 client.0 _ll_register_callbacks cb 0x7fd5ec00ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:52.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd5f67fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefc6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd5f67fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefc6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60558/0 (socket says 172.21.15.129:60558) 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd5f67fc640 1 -- 172.21.15.129:0/3867849021 learned_addr learned my addr 172.21.15.129:0/3867849021 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.964+0000 7fd5f77fe640 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefc7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.965+0000 7fd5f67fc640 1 -- 172.21.15.129:0/3867849021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb7a80 msgr2=0x5637aefc6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.965+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb7a80 0x5637aefc6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.965+0000 7fd5f67fc640 1 -- 172.21.15.129:0/3867849021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 msgr2=0x5637aefc7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.965+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefc7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.965+0000 7fd5f67fc640 1 -- 172.21.15.129:0/3867849021 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637aefc8140 con 0x5637aefbd5e0 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.965+0000 7fd5f77fe640 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefc7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.965+0000 7fd5f67fc640 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefc6bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5637aefd0a20 tx=0x7fd5f0004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f03a1e932e85fff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:52.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.974+0000 7fd5daff5640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:20:52.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.974+0000 7fd5daff5640 1 -- 172.21.15.129:0/3867849021 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0028040 con 0x5637aefbd5e0 2024-04-26T21:20:52.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.974+0000 7fd60027d740 1 -- 172.21.15.129:0/3867849021 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ec00a7e0 con 0x5637aefbd5e0 2024-04-26T21:20:52.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.974+0000 7fd5daff5640 1 -- 172.21.15.129:0/3867849021 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd5f00281e0 con 0x5637aefbd5e0 2024-04-26T21:20:52.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.976+0000 7fd5daff5640 1 -- 172.21.15.129:0/3867849021 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd5f00283e0 con 0x5637aefbd5e0 2024-04-26T21:20:52.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.977+0000 7fd60027d740 10 client.25340 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:52.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.977+0000 7fd60027d740 1 -- 172.21.15.129:0/3867849021 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5637aefc9550 con 0x5637aefbd5e0 2024-04-26T21:20:52.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.977+0000 7fd5daff5640 1 -- 172.21.15.129:0/3867849021 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ec00a7e0 con 0x5637aefbd5e0 2024-04-26T21:20:52.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.978+0000 7fd5daff5640 1 -- 172.21.15.129:0/3867849021 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fd5f00279d0 con 0x5637aefbd5e0 2024-04-26T21:20:52.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.978+0000 7fd60027d740 10 client.25340 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:52.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.978+0000 7fd60027d740 10 client.25340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:52.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.978+0000 7fd60027d740 20 client.25340 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:52.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.978+0000 7fd60027d740 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5637af0457b0 0x5637af047bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.978+0000 7fd60027d740 4 client.25340 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:20:52.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.978+0000 7fd60027d740 1 -- 172.21.15.129:0/3867849021 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5637aef24580 con 0x5637af0457b0 2024-04-26T21:20:52.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.979+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5637af0457b0 0x5637af047bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.979+0000 7fd5f6ffd640 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5637af0457b0 0x5637af047bf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f1b540bb9f126a server_cookie=b2adad1205c701ac in_seq=0 out_seq=0 2024-04-26T21:20:52.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:52.979+0000 7fd5daff5640 10 client.25340 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd5daff5640 1 -- 172.21.15.129:0/3867849021 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5637aef24580 con 0x5637af0457b0 2024-04-26T21:20:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd5daff5640 10 client.25340 handle_command_reply: tid=0 2024-04-26T21:20:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 client.25340 shutdown 2024-04-26T21:20:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 -- 172.21.15.129:0/3867849021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 msgr2=0x5637aefc6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefc6bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5637aefd0a20 tx=0x7fd5f0004a80 comp rx=0 tx=0).stop 2024-04-26T21:20:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 -- 172.21.15.129:0/3867849021 shutdown_connections 2024-04-26T21:20:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5637af0457b0 0x5637af047bf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5637aefbd5e0 0x5637aefc6bb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637aefc24f0 0x5637aefc7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:53.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 --2- 172.21.15.129:0/3867849021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5637aefb7a80 0x5637aefc6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:53.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd5daff5640 0 client.25340 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:53.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 -- 172.21.15.129:0/3867849021 >> 172.21.15.129:0/3867849021 conn(0x5637aefb30f0 msgr2=0x5637aefb1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:53.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.619+0000 7fd60027d740 1 -- 172.21.15.129:0/3867849021 shutdown_connections 2024-04-26T21:20:53.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.620+0000 7fd60027d740 1 -- 172.21.15.129:0/3867849021 wait complete. 2024-04-26T21:20:53.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.620+0000 7fd60027d740 20 client.25340 trim_cache size 0 max 16384 2024-04-26T21:20:53.622 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:20:53.622 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:20:53.622 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "eeb2d312-0673-43b5-9ab4-2b9ed46fce4a", 2024-04-26T21:20:53.623 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:20:53.623 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:20:53.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.635+0000 7fd5fd567640 1 -- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5e00a3d00 msgr2=0x7fd5e00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:53.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.635+0000 7fd5fd567640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5e00a3d00 0x7fd5e00a61c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000b940 tx=0x7fd5f000e040 comp rx=0 tx=0).stop 2024-04-26T21:20:53.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.635+0000 7fd5fd567640 1 -- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f810ad70 msgr2=0x7fd5f810c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:53.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.635+0000 7fd5fd567640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f810ad70 0x7fd5f810c260 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800a340 tx=0x7fd5e8030c50 comp rx=0 tx=0).stop 2024-04-26T21:20:53.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.636+0000 7fd5fd567640 1 -- 172.21.15.129:0/1863034918 shutdown_connections 2024-04-26T21:20:53.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.636+0000 7fd5fd567640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd5e00a3d00 0x7fd5e00a61c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:53.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.636+0000 7fd5fd567640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5f810ad70 0x7fd5f810c260 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:53.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.636+0000 7fd5fd567640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5f810a2f0 0x7fd5f810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:53.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.636+0000 7fd5fd567640 1 --2- 172.21.15.129:0/1863034918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f8109930 0x7fd5f810d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:53.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.636+0000 7fd5fd567640 1 -- 172.21.15.129:0/1863034918 >> 172.21.15.129:0/1863034918 conn(0x7fd5f806f840 msgr2=0x7fd5f810fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:53.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.640+0000 7fd5fd567640 1 -- 172.21.15.129:0/1863034918 shutdown_connections 2024-04-26T21:20:53.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:53.640+0000 7fd5fd567640 1 -- 172.21.15.129:0/1863034918 wait complete. 2024-04-26T21:20:53.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:20:53.675 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:20:53.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:53 smithi129 ceph-mon[29237]: pgmap v437: 97 pgs: 97 active+clean; 295 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 534 op/s 2024-04-26T21:20:53.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:53 smithi171 ceph-mon[33478]: pgmap v437: 97 pgs: 97 active+clean; 295 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 534 op/s 2024-04-26T21:20:53.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:53 smithi150 ceph-mon[32290]: pgmap v437: 97 pgs: 97 active+clean; 295 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 534 op/s 2024-04-26T21:20:54.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.056+0000 7ffb6759e640 1 -- 172.21.15.129:0/371553156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60099900 msgr2=0x7ffb60099d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.056+0000 7ffb6759e640 1 --2- 172.21.15.129:0/371553156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60099900 0x7ffb60099d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb6806c320 tx=0x7ffb68097050 comp rx=0 tx=0).stop 2024-04-26T21:20:54.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.056+0000 7ffb6759e640 1 -- 172.21.15.129:0/371553156 shutdown_connections 2024-04-26T21:20:54.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.056+0000 7ffb6759e640 1 --2- 172.21.15.129:0/371553156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60099900 0x7ffb60099d80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.056+0000 7ffb6759e640 1 --2- 172.21.15.129:0/371553156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb60096f50 0x7ffb600993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.056+0000 7ffb6759e640 1 --2- 172.21.15.129:0/371553156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb600945e0 0x7ffb60096a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.056+0000 7ffb6759e640 1 -- 172.21.15.129:0/371553156 >> 172.21.15.129:0/371553156 conn(0x7ffb600901c0 msgr2=0x7ffb60092620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.056+0000 7ffb6759e640 1 -- 172.21.15.129:0/371553156 shutdown_connections 2024-04-26T21:20:54.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.056+0000 7ffb6759e640 1 -- 172.21.15.129:0/371553156 wait complete. 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb6759e640 1 Processor -- start 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb6759e640 1 -- start start 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb6759e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb600945e0 0x7ffb60132480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb6759e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60096f50 0x7ffb601329c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb6759e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb60099900 0x7ffb60139a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb6759e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb600a7e70 con 0x7ffb600945e0 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb6759e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb600a7cf0 con 0x7ffb60096f50 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb6759e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffb600a7ff0 con 0x7ffb60099900 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb65d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60096f50 0x7ffb601329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb65d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60096f50 0x7ffb601329c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60586/0 (socket says 172.21.15.129:60586) 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb65d9b640 1 -- 172.21.15.129:0/2902353248 learned_addr learned my addr 172.21.15.129:0/2902353248 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:54.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb60099900 0x7ffb60139a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.058+0000 7ffb6659c640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb600945e0 0x7ffb60132480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb65d9b640 1 -- 172.21.15.129:0/2902353248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb60099900 msgr2=0x7ffb60139a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb65d9b640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb60099900 0x7ffb60139a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb65d9b640 1 -- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb600945e0 msgr2=0x7ffb60132480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb65d9b640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb600945e0 0x7ffb60132480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb65d9b640 1 -- 172.21.15.129:0/2902353248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6013a0a0 con 0x7ffb60096f50 2024-04-26T21:20:54.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb6659c640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb600945e0 0x7ffb60132480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:54.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb65d9b640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60096f50 0x7ffb601329c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e970 tx=0x7ffb5c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb577fe640 1 -- 172.21.15.129:0/2902353248 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00ccc0 con 0x7ffb60096f50 2024-04-26T21:20:54.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb577fe640 1 -- 172.21.15.129:0/2902353248 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb5c0108e0 con 0x7ffb60096f50 2024-04-26T21:20:54.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb6759e640 1 -- 172.21.15.129:0/2902353248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6013a390 con 0x7ffb60096f50 2024-04-26T21:20:54.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.059+0000 7ffb577fe640 1 -- 172.21.15.129:0/2902353248 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0187e0 con 0x7ffb60096f50 2024-04-26T21:20:54.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.060+0000 7ffb6759e640 1 -- 172.21.15.129:0/2902353248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6013a930 con 0x7ffb60096f50 2024-04-26T21:20:54.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.064+0000 7ffb6f809740 1 Processor -- start 2024-04-26T21:20:54.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.064+0000 7ffb6f809740 1 -- start start 2024-04-26T21:20:54.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.064+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edcd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.064+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edce280 0x56267edce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.064+0000 7ffb577fe640 1 -- 172.21.15.129:0/2902353248 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ffb5c010430 con 0x7ffb60096f50 2024-04-26T21:20:54.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edce280 0x56267edce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edcd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edcd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60596/0 (socket says 172.21.15.129:60596) 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edce280 0x56267edce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59874/0 (socket says 172.21.15.129:59874) 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb66d9d640 1 -- 172.21.15.129:0/1159189255 learned_addr learned my addr 172.21.15.129:0/1159189255 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6f809740 1 --2- 172.21.15.129:0/1159189255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edcec40 0x56267edd54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6f809740 1 -- 172.21.15.129:0/1159189255 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56267ed31d80 con 0x56267edce280 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6f809740 1 -- 172.21.15.129:0/1159189255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56267eceb8b0 con 0x56267edcd0b0 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6f809740 1 -- 172.21.15.129:0/1159189255 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56267ed2dd40 con 0x56267edcec40 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6659c640 1 -- 172.21.15.129:0/1159189255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edcec40 msgr2=0x56267edd54d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6659c640 1 --2- 172.21.15.129:0/1159189255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edcec40 0x56267edd54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6659c640 1 -- 172.21.15.129:0/1159189255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 msgr2=0x56267edcd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.065+0000 7ffb6659c640 1 --2- 172.21.15.129:0/1159189255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edcd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.066+0000 7ffb6659c640 1 -- 172.21.15.129:0/1159189255 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56267ecbca40 con 0x56267edce280 2024-04-26T21:20:54.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.066+0000 7ffb577fe640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ffb500a3d40 0x7ffb500a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.066+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ffb500a3d40 0x7ffb500a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.066+0000 7ffb577fe640 1 -- 172.21.15.129:0/2902353248 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ffb5c0d0dd0 con 0x7ffb60096f50 2024-04-26T21:20:54.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.066+0000 7ffb6659c640 1 --2- 172.21.15.129:0/1159189255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edce280 0x56267edce700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800bcd0 tx=0x7ffb5800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8388b7037b2e03fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.067+0000 7ffb54ff9640 1 -- 172.21.15.129:0/1159189255 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58015070 con 0x56267edce280 2024-04-26T21:20:54.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.067+0000 7ffb54ff9640 1 -- 172.21.15.129:0/1159189255 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb5801a910 con 0x56267edce280 2024-04-26T21:20:54.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.067+0000 7ffb54ff9640 1 -- 172.21.15.129:0/1159189255 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58011980 con 0x56267edce280 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.068+0000 7ffb6f809740 1 -- 172.21.15.129:0/1159189255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edce280 msgr2=0x56267edce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.068+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ffb500a3d40 0x7ffb500a6200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffb68065e50 tx=0x7ffb68068800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.068+0000 7ffb6f809740 1 --2- 172.21.15.129:0/1159189255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edce280 0x56267edce700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800bcd0 tx=0x7ffb5800f730 comp rx=0 tx=0).stop 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.068+0000 7ffb6f809740 1 -- 172.21.15.129:0/1159189255 shutdown_connections 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.068+0000 7ffb6f809740 1 --2- 172.21.15.129:0/1159189255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edcec40 0x56267edd54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.068+0000 7ffb6f809740 1 --2- 172.21.15.129:0/1159189255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edce280 0x56267edce700 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.068+0000 7ffb6f809740 1 --2- 172.21.15.129:0/1159189255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edcd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.069+0000 7ffb6f809740 1 -- 172.21.15.129:0/1159189255 >> 172.21.15.129:0/1159189255 conn(0x56267edc85a0 msgr2=0x56267edc89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.069+0000 7ffb6f809740 1 -- 172.21.15.129:0/1159189255 shutdown_connections 2024-04-26T21:20:54.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.069+0000 7ffb6f809740 1 -- 172.21.15.129:0/1159189255 wait complete. 2024-04-26T21:20:54.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.070+0000 7ffb6f809740 1 -- start start 2024-04-26T21:20:54.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.070+0000 7ffb6f809740 1 Processor -- start 2024-04-26T21:20:54.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcd0b0 0x56267edd73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edd7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcd0b0 0x56267edd73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcd0b0 0x56267edd73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59880/0 (socket says 172.21.15.129:59880) 2024-04-26T21:20:54.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcec40 0x56267edde940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb65d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edd7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb65d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edd7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:55252/0 (socket says 172.21.15.129:55252) 2024-04-26T21:20:54.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb6659c640 1 -- 172.21.15.129:0/2620055199 learned_addr learned my addr 172.21.15.129:0/2620055199 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:54.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56267edd5c50 con 0x56267edcd0b0 2024-04-26T21:20:54.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcec40 0x56267edde940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.071+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56267ecb8d10 con 0x56267edcec40 2024-04-26T21:20:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.072+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56267edd5dd0 con 0x56267edce280 2024-04-26T21:20:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.072+0000 7ffb6f809740 10 client.0 _ll_register_callbacks cb 0x56267ece5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.073+0000 7ffb6659c640 1 -- 172.21.15.129:0/2620055199 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 msgr2=0x56267edd7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.073+0000 7ffb6659c640 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edd7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.073+0000 7ffb6659c640 1 -- 172.21.15.129:0/2620055199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcec40 msgr2=0x56267edde940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.073+0000 7ffb6659c640 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcec40 0x56267edde940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.073+0000 7ffb6659c640 1 -- 172.21.15.129:0/2620055199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56267eb0d7d0 con 0x56267edcd0b0 2024-04-26T21:20:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.073+0000 7ffb65d9b640 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edd7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.073+0000 7ffb6659c640 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcd0b0 0x56267edd73c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800c010 tx=0x7ffb5801a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a1e6b5c1df4256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.073+0000 7ffb4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:20:54.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.073+0000 7ffb4dffb640 1 -- 172.21.15.129:0/2620055199 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58028060 con 0x56267edcd0b0 2024-04-26T21:20:54.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.074+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb5801a910 con 0x56267edcd0b0 2024-04-26T21:20:54.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.074+0000 7ffb4dffb640 1 -- 172.21.15.129:0/2620055199 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ffb58005040 con 0x56267edcd0b0 2024-04-26T21:20:54.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.074+0000 7ffb4dffb640 1 -- 172.21.15.129:0/2620055199 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ffb58015040 con 0x56267edcd0b0 2024-04-26T21:20:54.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.074+0000 7ffb6f809740 10 client.15756 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:54.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.074+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56267ea433f0 con 0x56267edcd0b0 2024-04-26T21:20:54.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.074+0000 7ffb4dffb640 1 -- 172.21.15.129:0/2620055199 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb58011700 con 0x56267edcd0b0 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb4dffb640 1 -- 172.21.15.129:0/2620055199 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ffb58022030 con 0x56267edcd0b0 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb6f809740 10 client.15756 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb6f809740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb6f809740 20 client.15756 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb6f809740 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56267ede2a80 0x56267ede4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb6f809740 4 client.15756 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffb5801a910 con 0x56267ede2a80 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb65d9b640 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56267ede2a80 0x56267ede4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb4dffb640 10 client.15756 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:54.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.075+0000 7ffb65d9b640 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56267ede2a80 0x56267ede4e70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788f000e065ef1f4 server_cookie=bb5449c0c81de222 in_seq=0 out_seq=0 2024-04-26T21:20:54.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb4dffb640 1 -- 172.21.15.129:0/2620055199 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ffb5801a910 con 0x56267ede2a80 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb4dffb640 10 client.15756 handle_command_reply: tid=0 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb6f809740 1 client.15756 shutdown 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcd0b0 msgr2=0x56267edd73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb6f809740 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcd0b0 0x56267edd73c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800c010 tx=0x7ffb5801a690 comp rx=0 tx=0).stop 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 shutdown_connections 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb6f809740 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56267ede2a80 0x56267ede4e70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb6f809740 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcec40 0x56267edde940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb4dffb640 0 client.15756 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb6f809740 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edd7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb6f809740 1 --2- 172.21.15.129:0/2620055199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcd0b0 0x56267edd73c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.078+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 >> 172.21.15.129:0/2620055199 conn(0x56267edc85a0 msgr2=0x56267edc76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.079+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 shutdown_connections 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.079+0000 7ffb6f809740 1 -- 172.21.15.129:0/2620055199 wait complete. 2024-04-26T21:20:54.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.079+0000 7ffb6f809740 20 client.15756 trim_cache size 0 max 16384 2024-04-26T21:20:54.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.097+0000 7ffb6f809740 1 Processor -- start 2024-04-26T21:20:54.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.097+0000 7ffb6f809740 1 -- start start 2024-04-26T21:20:54.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edd69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edde900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267edd6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb6f809740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601416f0 con 0x56267edcec40 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb6f809740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60143f70 con 0x56267edcd0b0 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb6f809740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60141d60 con 0x56267edce280 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267edd6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267edd6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59894/0 (socket says 172.21.15.129:59894) 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb66d9d640 1 -- 172.21.15.129:0/3312173411 learned_addr learned my addr 172.21.15.129:0/3312173411 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edd69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb65d9b640 1 --2- 172.21.15.129:0/3312173411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edde900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb66d9d640 1 -- 172.21.15.129:0/3312173411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 msgr2=0x56267edde900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/3312173411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edde900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb66d9d640 1 -- 172.21.15.129:0/3312173411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 msgr2=0x56267edd69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/3312173411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edd69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb66d9d640 1 -- 172.21.15.129:0/3312173411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56267ea030f0 con 0x56267edcec40 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb6659c640 1 --2- 172.21.15.129:0/3312173411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edd69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/3312173411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267edd6f20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffb68097640 tx=0x7ffb680a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585c0ef8a649b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.098+0000 7ffb4ffff640 1 -- 172.21.15.129:0/3312173411 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68076660 con 0x56267edcec40 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb4ffff640 1 -- 172.21.15.129:0/3312173411 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb6809b040 con 0x56267edcec40 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb4ffff640 1 -- 172.21.15.129:0/3312173411 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680adb60 con 0x56267edcec40 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb6f809740 1 -- 172.21.15.129:0/3312173411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 msgr2=0x56267edd6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3312173411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267edd6f20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffb68097640 tx=0x7ffb680a9000 comp rx=0 tx=0).stop 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb6f809740 1 -- 172.21.15.129:0/3312173411 shutdown_connections 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3312173411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267edd6f20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3312173411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edce280 0x56267edde900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3312173411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edcd0b0 0x56267edd69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb6f809740 1 -- 172.21.15.129:0/3312173411 >> 172.21.15.129:0/3312173411 conn(0x56267edc85a0 msgr2=0x56267edc76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb6f809740 1 -- 172.21.15.129:0/3312173411 shutdown_connections 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.099+0000 7ffb6f809740 1 -- 172.21.15.129:0/3312173411 wait complete. 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6f809740 1 -- start start 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6f809740 1 Processor -- start 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edcd0b0 0x56267eddcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edce280 0x56267eddcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6f809740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267eddd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6f809740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60142430 con 0x56267edcec40 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6f809740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601438a0 con 0x56267edce280 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6f809740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60142b00 con 0x56267edcd0b0 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6f809740 10 client.0 _ll_register_callbacks cb 0x7ffb68050d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267eddd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267eddd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59906/0 (socket says 172.21.15.129:59906) 2024-04-26T21:20:54.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb66d9d640 1 -- 172.21.15.129:0/3974659979 learned_addr learned my addr 172.21.15.129:0/3974659979 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb65d9b640 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edce280 0x56267eddcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb6659c640 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edcd0b0 0x56267eddcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb66d9d640 1 -- 172.21.15.129:0/3974659979 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edcd0b0 msgr2=0x56267eddcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edcd0b0 0x56267eddcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb66d9d640 1 -- 172.21.15.129:0/3974659979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edce280 msgr2=0x56267eddcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edce280 0x56267eddcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb66d9d640 1 -- 172.21.15.129:0/3974659979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56267ecf24b0 con 0x56267edcec40 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.100+0000 7ffb65d9b640 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edce280 0x56267eddcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb66d9d640 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267eddd660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb68052070 tx=0x7ffb6809eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda0a96487223440 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb4dffb640 1 -- 172.21.15.129:0/3974659979 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680adaa0 con 0x56267edcec40 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb4dffb640 1 -- 172.21.15.129:0/3974659979 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ffb680adc40 con 0x56267edcec40 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb6f809740 1 -- 172.21.15.129:0/3974659979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb6809b040 con 0x56267edcec40 2024-04-26T21:20:54.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb4dffb640 1 -- 172.21.15.129:0/3974659979 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ffb68076660 con 0x56267edcec40 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb6f809740 10 client.15762 fetch_fsmap learned FSMap version 19 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb6f809740 1 -- 172.21.15.129:0/3974659979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56267ee48ff0 con 0x56267edcec40 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb4dffb640 1 -- 172.21.15.129:0/3974659979 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb6809b040 con 0x56267edcec40 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb4dffb640 1 -- 172.21.15.129:0/3974659979 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ffb680a04f0 con 0x56267edcec40 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb6f809740 10 client.15762 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb6f809740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb6f809740 20 client.15762 populate_metadata read hostname 'smithi129' 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56267ee499a0 0x56267ee4bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb6f809740 4 client.15762 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.101+0000 7ffb6f809740 1 -- 172.21.15.129:0/3974659979 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56267ed3f990 con 0x56267ee499a0 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.102+0000 7ffb6659c640 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56267ee499a0 0x56267ee4bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.102+0000 7ffb4dffb640 10 client.15762 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:54.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.102+0000 7ffb6659c640 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56267ee499a0 0x56267ee4bdc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba86ca20e469a70 server_cookie=1561eac949735255 in_seq=0 out_seq=0 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb4dffb640 1 -- 172.21.15.129:0/3974659979 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x56267ed3f990 con 0x56267ee499a0 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb4dffb640 10 client.15762 handle_command_reply: tid=0 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 client.15762 shutdown 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 -- 172.21.15.129:0/3974659979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 msgr2=0x56267eddd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267eddd660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb68052070 tx=0x7ffb6809eeb0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 -- 172.21.15.129:0/3974659979 shutdown_connections 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56267ee499a0 0x56267ee4bdc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56267edce280 0x56267eddcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56267edcec40 0x56267eddd660 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb4dffb640 0 client.15762 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 --2- 172.21.15.129:0/3974659979 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56267edcd0b0 0x56267eddcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 -- 172.21.15.129:0/3974659979 >> 172.21.15.129:0/3974659979 conn(0x56267edc85a0 msgr2=0x56267edcb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 -- 172.21.15.129:0/3974659979 shutdown_connections 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 1 -- 172.21.15.129:0/3974659979 wait complete. 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.107+0000 7ffb6f809740 20 client.15762 trim_cache size 0 max 16384 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (3 inodes in the stack)", 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stdout: "eeb2d312-0673-43b5-9ab4-2b9ed46fce4a": { 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "eeb2d312-0673-43b5-9ab4-2b9ed46fce4a", 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:20:54.111 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:20:54.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:20:54.112 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:20:54.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 -- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ffb500a3d40 msgr2=0x7ffb500a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ffb500a3d40 0x7ffb500a6200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffb68065e50 tx=0x7ffb68068800 comp rx=0 tx=0).stop 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 -- 172.21.15.129:0/2902353248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60096f50 msgr2=0x7ffb601329c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60096f50 0x7ffb601329c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e970 tx=0x7ffb5c00ee30 comp rx=0 tx=0).stop 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 -- 172.21.15.129:0/2902353248 shutdown_connections 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ffb500a3d40 0x7ffb500a6200 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb60096f50 0x7ffb601329c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb60099900 0x7ffb60139a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 --2- 172.21.15.129:0/2902353248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb600945e0 0x7ffb60132480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 -- 172.21.15.129:0/2902353248 >> 172.21.15.129:0/2902353248 conn(0x7ffb600901c0 msgr2=0x7ffb60094f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 -- 172.21.15.129:0/2902353248 shutdown_connections 2024-04-26T21:20:54.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:20:54.108+0000 7ffb6759e640 1 -- 172.21.15.129:0/2902353248 wait complete. 2024-04-26T21:20:54.121 INFO:tasks.cephfs.filesystem:scrub status for tag:eeb2d312-0673-43b5-9ab4-2b9ed46fce4a - {'path': '/', 'tag': 'eeb2d312-0673-43b5-9ab4-2b9ed46fce4a', 'options': 'recursive,force'} 2024-04-26T21:20:54.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:54 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:20:54.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:54 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:20:54.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:54 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:20:54.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:54 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:20:54.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:54 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:20:54.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:54 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:20:54.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:54 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:20:54.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:54 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:20:54.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:54 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:20:55.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:55 smithi129 ceph-mon[29237]: pgmap v438: 97 pgs: 97 active+clean; 297 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-04-26T21:20:55.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:55 smithi171 ceph-mon[33478]: pgmap v438: 97 pgs: 97 active+clean; 297 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-04-26T21:20:55.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:55 smithi150 ceph-mon[32290]: pgmap v438: 97 pgs: 97 active+clean; 297 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-04-26T21:20:58.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:58 smithi129 ceph-mon[29237]: pgmap v439: 97 pgs: 97 active+clean; 300 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 225 op/s 2024-04-26T21:20:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:58 smithi171 ceph-mon[33478]: pgmap v439: 97 pgs: 97 active+clean; 300 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 225 op/s 2024-04-26T21:20:58.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:58 smithi150 ceph-mon[32290]: pgmap v439: 97 pgs: 97 active+clean; 300 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 225 op/s 2024-04-26T21:20:59.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:20:59 smithi129 ceph-mon[29237]: pgmap v440: 97 pgs: 97 active+clean; 303 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-04-26T21:20:59.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:20:59 smithi171 ceph-mon[33478]: pgmap v440: 97 pgs: 97 active+clean; 303 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-04-26T21:20:59.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:20:59 smithi150 ceph-mon[32290]: pgmap v440: 97 pgs: 97 active+clean; 303 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-04-26T21:21:01.768 INFO:tasks.workunit.client.0.smithi129.stdout: 72 1828042 571 1280098 483 965739 1360 2024-04-26T21:21:02.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:02 smithi129 ceph-mon[29237]: pgmap v441: 97 pgs: 97 active+clean; 305 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 123 op/s 2024-04-26T21:21:02.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:02 smithi171 ceph-mon[33478]: pgmap v441: 97 pgs: 97 active+clean; 305 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 123 op/s 2024-04-26T21:21:02.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:02 smithi150 ceph-mon[32290]: pgmap v441: 97 pgs: 97 active+clean; 305 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 123 op/s 2024-04-26T21:21:03.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:03 smithi129 ceph-mon[29237]: pgmap v442: 97 pgs: 97 active+clean; 305 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-04-26T21:21:03.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:03 smithi171 ceph-mon[33478]: pgmap v442: 97 pgs: 97 active+clean; 305 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-04-26T21:21:03.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:03 smithi150 ceph-mon[32290]: pgmap v442: 97 pgs: 97 active+clean; 305 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-04-26T21:21:06.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:06 smithi129 ceph-mon[29237]: pgmap v443: 97 pgs: 97 active+clean; 323 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 176 op/s 2024-04-26T21:21:06.743 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:06 smithi171 ceph-mon[33478]: pgmap v443: 97 pgs: 97 active+clean; 323 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 176 op/s 2024-04-26T21:21:06.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:06 smithi150 ceph-mon[32290]: pgmap v443: 97 pgs: 97 active+clean; 323 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 176 op/s 2024-04-26T21:21:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:08 smithi150 ceph-mon[32290]: pgmap v444: 97 pgs: 97 active+clean; 340 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-04-26T21:21:08.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:08 smithi129 ceph-mon[29237]: pgmap v444: 97 pgs: 97 active+clean; 340 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-04-26T21:21:08.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:08 smithi171 ceph-mon[33478]: pgmap v444: 97 pgs: 97 active+clean; 340 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-04-26T21:21:10.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:10 smithi129 ceph-mon[29237]: pgmap v445: 97 pgs: 97 active+clean; 372 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.4 MiB/s wr, 410 op/s 2024-04-26T21:21:10.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:10 smithi171 ceph-mon[33478]: pgmap v445: 97 pgs: 97 active+clean; 372 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.4 MiB/s wr, 410 op/s 2024-04-26T21:21:10.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:10 smithi150 ceph-mon[32290]: pgmap v445: 97 pgs: 97 active+clean; 372 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.4 MiB/s wr, 410 op/s 2024-04-26T21:21:11.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:11 smithi129 ceph-mon[29237]: pgmap v446: 97 pgs: 97 active+clean; 382 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 463 op/s 2024-04-26T21:21:11.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:11 smithi171 ceph-mon[33478]: pgmap v446: 97 pgs: 97 active+clean; 382 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 463 op/s 2024-04-26T21:21:11.768 INFO:tasks.workunit.client.0.smithi129.stdout: 74 950577 310 642553 239 500805 804 2024-04-26T21:21:11.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:11 smithi150 ceph-mon[32290]: pgmap v446: 97 pgs: 97 active+clean; 382 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 463 op/s 2024-04-26T21:21:14.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:14 smithi129 ceph-mon[29237]: pgmap v447: 97 pgs: 97 active+clean; 386 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 485 op/s 2024-04-26T21:21:14.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:14 smithi171 ceph-mon[33478]: pgmap v447: 97 pgs: 97 active+clean; 386 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 485 op/s 2024-04-26T21:21:14.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:14 smithi150 ceph-mon[32290]: pgmap v447: 97 pgs: 97 active+clean; 386 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 485 op/s 2024-04-26T21:21:16.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:16 smithi129 ceph-mon[29237]: pgmap v448: 97 pgs: 97 active+clean; 385 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 497 op/s 2024-04-26T21:21:16.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:16 smithi171 ceph-mon[33478]: pgmap v448: 97 pgs: 97 active+clean; 385 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 497 op/s 2024-04-26T21:21:16.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:16 smithi150 ceph-mon[32290]: pgmap v448: 97 pgs: 97 active+clean; 385 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 497 op/s 2024-04-26T21:21:17.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:17 smithi129 ceph-mon[29237]: pgmap v449: 97 pgs: 97 active+clean; 389 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 401 op/s 2024-04-26T21:21:17.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:17 smithi171 ceph-mon[33478]: pgmap v449: 97 pgs: 97 active+clean; 389 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 401 op/s 2024-04-26T21:21:17.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:17 smithi150 ceph-mon[32290]: pgmap v449: 97 pgs: 97 active+clean; 389 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 401 op/s 2024-04-26T21:21:19.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:19 smithi129 ceph-mon[29237]: pgmap v450: 97 pgs: 97 active+clean; 391 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 335 op/s 2024-04-26T21:21:19.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:19 smithi171 ceph-mon[33478]: pgmap v450: 97 pgs: 97 active+clean; 391 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 335 op/s 2024-04-26T21:21:19.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:19 smithi150 ceph-mon[32290]: pgmap v450: 97 pgs: 97 active+clean; 391 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 335 op/s 2024-04-26T21:21:21.768 INFO:tasks.workunit.client.0.smithi129.stdout: 81 1664102 424 1135740 320 889500 1564 2024-04-26T21:21:22.721 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:22 smithi129 ceph-mon[29237]: pgmap v451: 97 pgs: 97 active+clean; 395 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 180 op/s 2024-04-26T21:21:22.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:22 smithi171 ceph-mon[33478]: pgmap v451: 97 pgs: 97 active+clean; 395 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 180 op/s 2024-04-26T21:21:22.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:22 smithi150 ceph-mon[32290]: pgmap v451: 97 pgs: 97 active+clean; 395 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 180 op/s 2024-04-26T21:21:23.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:23 smithi129 ceph-mon[29237]: pgmap v452: 97 pgs: 97 active+clean; 394 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 134 op/s 2024-04-26T21:21:23.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:23 smithi171 ceph-mon[33478]: pgmap v452: 97 pgs: 97 active+clean; 394 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 134 op/s 2024-04-26T21:21:23.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:23 smithi150 ceph-mon[32290]: pgmap v452: 97 pgs: 97 active+clean; 394 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 134 op/s 2024-04-26T21:21:24.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:21:24.122 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:21:24.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.344+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1625229352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b84106580 msgr2=0x7f5b84106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.344+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1625229352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b84106580 0x7f5b84106a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009ca0 tx=0x7f5b7c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.344+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1625229352 shutdown_connections 2024-04-26T21:21:24.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.344+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1625229352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b84106580 0x7f5b84106a00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.344+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1625229352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b84103bd0 0x7f5b84106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.344+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1625229352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b84101260 0x7f5b84103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.344+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1625229352 >> 172.21.15.129:0/1625229352 conn(0x7f5b840fce20 msgr2=0x7f5b840ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.344+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1625229352 shutdown_connections 2024-04-26T21:21:24.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.344+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1625229352 wait complete. 2024-04-26T21:21:24.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.345+0000 7f5b8a90a640 1 Processor -- start 2024-04-26T21:21:24.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.345+0000 7f5b8a90a640 1 -- start start 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b8a90a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b84101260 0x7f5b84196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b8a90a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b84103bd0 0x7f5b84196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b8a90a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b84106580 0x7f5b8419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b8a90a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8410a320 con 0x7f5b84103bd0 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b8a90a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8410a1a0 con 0x7f5b84106580 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b8a90a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8410a4a0 con 0x7f5b84101260 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b837fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b84103bd0 0x7f5b84196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b88ea7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b84106580 0x7f5b8419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b83fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b84101260 0x7f5b84196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b837fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b84103bd0 0x7f5b84196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48784/0 (socket says 172.21.15.129:48784) 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b837fe640 1 -- 172.21.15.129:0/1771041023 learned_addr learned my addr 172.21.15.129:0/1771041023 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b837fe640 1 -- 172.21.15.129:0/1771041023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b84101260 msgr2=0x7f5b84196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b837fe640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b84101260 0x7f5b84196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b837fe640 1 -- 172.21.15.129:0/1771041023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b84106580 msgr2=0x7f5b8419df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b837fe640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b84106580 0x7f5b8419df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b837fe640 1 -- 172.21.15.129:0/1771041023 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8419e5a0 con 0x7f5b84103bd0 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.346+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b84106580 0x7f5b8419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.347+0000 7f5b837fe640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b84103bd0 0x7f5b84196ec0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5b7801fb40 tx=0x7f5b7801d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.347+0000 7f5b817fa640 1 -- 172.21.15.129:0/1771041023 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7801dd20 con 0x7f5b84103bd0 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.347+0000 7f5b817fa640 1 -- 172.21.15.129:0/1771041023 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b780218f0 con 0x7f5b84103bd0 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.347+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1771041023 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8419e890 con 0x7f5b84103bd0 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.347+0000 7f5b817fa640 1 -- 172.21.15.129:0/1771041023 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780298b0 con 0x7f5b84103bd0 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.347+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1771041023 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8419ee30 con 0x7f5b84103bd0 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.350+0000 7f5b817fa640 1 -- 172.21.15.129:0/1771041023 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5b78021440 con 0x7f5b84103bd0 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.351+0000 7f5b8d625740 1 Processor -- start 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.351+0000 7f5b8d625740 1 -- start start 2024-04-26T21:21:24.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.351+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.351+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee1a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.351+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee21300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.351+0000 7f5b8d625740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bdded7dd80 con 0x55bddee1a0b0 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.351+0000 7f5b8d625740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bdded378b0 con 0x55bddee1aa70 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.351+0000 7f5b8d625740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bdded79d40 con 0x55bddee18ee0 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b88ea7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b88ea7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee19300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34998/0 (socket says 172.21.15.129:34998) 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b88ea7640 1 -- 172.21.15.129:0/2024817510 learned_addr learned my addr 172.21.15.129:0/2024817510 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b83fff640 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee1a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b817fa640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b6c0a3cc0 0x7f5b6c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b88ea7640 1 -- 172.21.15.129:0/2024817510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 msgr2=0x55bddee21300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:24.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b837fe640 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee21300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b817fa640 1 -- 172.21.15.129:0/1771041023 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5b780e1870 con 0x7f5b84103bd0 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee21300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b88ea7640 1 -- 172.21.15.129:0/2024817510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 msgr2=0x55bddee1a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee1a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b88ea7640 1 -- 172.21.15.129:0/2024817510 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdded08a40 con 0x55bddee18ee0 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b83fff640 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee1a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b837fe640 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee21300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.352+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b6c0a3cc0 0x7f5b6c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.353+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee19300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c035000 tx=0x7f5b7c009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3ca2819ab1fcd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.353+0000 7f5b6a7fc640 1 -- 172.21.15.129:0/2024817510 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c043040 con 0x55bddee18ee0 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.353+0000 7f5b6a7fc640 1 -- 172.21.15.129:0/2024817510 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b7c036040 con 0x55bddee18ee0 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.353+0000 7f5b6a7fc640 1 -- 172.21.15.129:0/2024817510 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c02fa00 con 0x55bddee18ee0 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b8d625740 1 -- 172.21.15.129:0/2024817510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 msgr2=0x55bddee19300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b6c0a3cc0 0x7f5b6c0a6180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c04c000 tx=0x7f5b7c04d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b8d625740 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee19300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c035000 tx=0x7f5b7c009800 comp rx=0 tx=0).stop 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b8d625740 1 -- 172.21.15.129:0/2024817510 shutdown_connections 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b8d625740 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee21300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b8d625740 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee1a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b8d625740 1 --2- 172.21.15.129:0/2024817510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee19300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b8d625740 1 -- 172.21.15.129:0/2024817510 >> 172.21.15.129:0/2024817510 conn(0x55bddee143d0 msgr2=0x55bddee147d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b8d625740 1 -- 172.21.15.129:0/2024817510 shutdown_connections 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.354+0000 7f5b8d625740 1 -- 172.21.15.129:0/2024817510 wait complete. 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b8d625740 1 -- start start 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b8d625740 1 Processor -- start 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee18ee0 0x55bddee23330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1a0b0 0x55bddee23870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee1aa70 0x55bddee2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b8d625740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bddee21a20 con 0x55bddee18ee0 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b8d625740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bdded04d10 con 0x55bddee1a0b0 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b8d625740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bddee21ba0 con 0x55bddee1aa70 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b8d625740 10 client.0 _ll_register_callbacks cb 0x55bdded31210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b83fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee18ee0 0x55bddee23330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b837fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1a0b0 0x55bddee23870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b88ea7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee1aa70 0x55bddee2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b837fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1a0b0 0x55bddee23870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:48010/0 (socket says 172.21.15.129:48010) 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.355+0000 7f5b837fe640 1 -- 172.21.15.129:0/3260920147 learned_addr learned my addr 172.21.15.129:0/3260920147 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b837fe640 1 -- 172.21.15.129:0/3260920147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee1aa70 msgr2=0x55bddee2a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b837fe640 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee1aa70 0x55bddee2a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b837fe640 1 -- 172.21.15.129:0/3260920147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee18ee0 msgr2=0x55bddee23330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b837fe640 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee18ee0 0x55bddee23330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b837fe640 1 -- 172.21.15.129:0/3260920147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bddeb597d0 con 0x55bddee1a0b0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee1aa70 0x55bddee2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b837fe640 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1a0b0 0x55bddee23870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b780aa460 tx=0x7f5b7807d3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a192485b0f522e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b637fe640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b637fe640 1 -- 172.21.15.129:0/3260920147 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780b1d90 con 0x55bddee1a0b0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b637fe640 1 -- 172.21.15.129:0/3260920147 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5b78021aa0 con 0x55bddee1a0b0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b8d625740 1 -- 172.21.15.129:0/3260920147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7c036040 con 0x55bddee1a0b0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b637fe640 1 -- 172.21.15.129:0/3260920147 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5b780b6650 con 0x55bddee1a0b0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b8d625740 10 client.25367 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b8d625740 1 -- 172.21.15.129:0/3260920147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bddea8f3f0 con 0x55bddee1a0b0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.356+0000 7f5b637fe640 1 -- 172.21.15.129:0/3260920147 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b7c036040 con 0x55bddee1a0b0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b637fe640 1 -- 172.21.15.129:0/3260920147 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f5b780e1ce0 con 0x55bddee1a0b0 2024-04-26T21:21:24.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b8d625740 10 client.25367 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:24.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b8d625740 10 client.25367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:24.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b8d625740 20 client.25367 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:24.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bddee2ed60 0x55bddee31170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b8d625740 4 client.25367 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:21:24.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b8d625740 1 -- 172.21.15.129:0/3260920147 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdded8b990 con 0x55bddee2ed60 2024-04-26T21:21:24.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b83fff640 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bddee2ed60 0x55bddee31170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b637fe640 10 client.25367 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:24.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.357+0000 7f5b83fff640 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bddee2ed60 0x55bddee31170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbddb1b60042e73 server_cookie=fe9df36b6d3c36e2 in_seq=0 out_seq=0 2024-04-26T21:21:24.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.358+0000 7f5b637fe640 1 -- 172.21.15.129:0/3260920147 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdded8b990 con 0x55bddee2ed60 2024-04-26T21:21:24.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.358+0000 7f5b637fe640 10 client.25367 handle_command_reply: tid=0 2024-04-26T21:21:24.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.358+0000 7f5b8d625740 1 client.25367 shutdown 2024-04-26T21:21:24.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 -- 172.21.15.129:0/3260920147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1a0b0 msgr2=0x55bddee23870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1a0b0 0x55bddee23870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b780aa460 tx=0x7f5b7807d3c0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 -- 172.21.15.129:0/3260920147 shutdown_connections 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bddee2ed60 0x55bddee31170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1a0b0 0x55bddee23870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b637fe640 0 client.25367 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee1aa70 0x55bddee2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3260920147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee18ee0 0x55bddee23330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 -- 172.21.15.129:0/3260920147 >> 172.21.15.129:0/3260920147 conn(0x55bddee143d0 msgr2=0x55bddee13550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 -- 172.21.15.129:0/3260920147 shutdown_connections 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 1 -- 172.21.15.129:0/3260920147 wait complete. 2024-04-26T21:21:24.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.359+0000 7f5b8d625740 20 client.25367 trim_cache size 0 max 16384 2024-04-26T21:21:24.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b8d625740 1 Processor -- start 2024-04-26T21:21:24.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b8d625740 1 -- start start 2024-04-26T21:21:24.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee2a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee1db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b8d625740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841a5be0 con 0x55bddee1a0b0 2024-04-26T21:21:24.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b8d625740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841a8460 con 0x55bddee1aa70 2024-04-26T21:21:24.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b8d625740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841a6250 con 0x55bddee18ee0 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b88ea7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee1db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b88ea7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee1db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:48014/0 (socket says 172.21.15.129:48014) 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b88ea7640 1 -- 172.21.15.129:0/31249264 learned_addr learned my addr 172.21.15.129:0/31249264 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b83fff640 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b837fe640 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee2a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b88ea7640 1 -- 172.21.15.129:0/31249264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 msgr2=0x55bddee211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b88ea7640 1 -- 172.21.15.129:0/31249264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 msgr2=0x55bddee2a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee2a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b88ea7640 1 -- 172.21.15.129:0/31249264 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bddea4f0f0 con 0x55bddee1aa70 2024-04-26T21:21:24.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.377+0000 7f5b837fe640 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee2a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:24.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.378+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee1db90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c035000 tx=0x7f5b7c040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3e88e68621b1761 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.378+0000 7f5b69ffb640 1 -- 172.21.15.129:0/31249264 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c043030 con 0x55bddee1aa70 2024-04-26T21:21:24.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.378+0000 7f5b69ffb640 1 -- 172.21.15.129:0/31249264 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b7c036030 con 0x55bddee1aa70 2024-04-26T21:21:24.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.378+0000 7f5b69ffb640 1 -- 172.21.15.129:0/31249264 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c056040 con 0x55bddee1aa70 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 -- 172.21.15.129:0/31249264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 msgr2=0x55bddee1db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee1db90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c035000 tx=0x7f5b7c040000 comp rx=0 tx=0).stop 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 -- 172.21.15.129:0/31249264 shutdown_connections 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddee1db90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee2a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 --2- 172.21.15.129:0/31249264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 -- 172.21.15.129:0/31249264 >> 172.21.15.129:0/31249264 conn(0x55bddee143d0 msgr2=0x55bddee0f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 -- 172.21.15.129:0/31249264 shutdown_connections 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 -- 172.21.15.129:0/31249264 wait complete. 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 -- start start 2024-04-26T21:21:24.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.379+0000 7f5b8d625740 1 Processor -- start 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee27b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b8d625740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddeea42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b8d625740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841a6920 con 0x55bddee1a0b0 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b8d625740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841a7d90 con 0x55bddee1aa70 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b8d625740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841a6ff0 con 0x55bddee18ee0 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b8d625740 10 client.0 _ll_register_callbacks cb 0x7f5b780a9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b88ea7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddeea42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b88ea7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddeea42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:48022/0 (socket says 172.21.15.129:48022) 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b88ea7640 1 -- 172.21.15.129:0/3563632557 learned_addr learned my addr 172.21.15.129:0/3563632557 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b837fe640 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee27b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b83fff640 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b88ea7640 1 -- 172.21.15.129:0/3563632557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 msgr2=0x55bddee275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b88ea7640 1 -- 172.21.15.129:0/3563632557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 msgr2=0x55bddee27b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee27b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b88ea7640 1 -- 172.21.15.129:0/3563632557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdded3e4b0 con 0x55bddee1aa70 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.380+0000 7f5b88ea7640 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddeea42b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c031b50 tx=0x7f5b7c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2162e295c088a542 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b637fe640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b637fe640 1 -- 172.21.15.129:0/3563632557 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c043040 con 0x55bddee1aa70 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b637fe640 1 -- 172.21.15.129:0/3563632557 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5b7c009800 con 0x55bddee1aa70 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b8d625740 1 -- 172.21.15.129:0/3563632557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7c036030 con 0x55bddee1aa70 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b637fe640 1 -- 172.21.15.129:0/3563632557 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5b7c004040 con 0x55bddee1aa70 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b8d625740 10 client.25373 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b8d625740 1 -- 172.21.15.129:0/3563632557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bddeea5670 con 0x55bddee1aa70 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b637fe640 1 -- 172.21.15.129:0/3563632557 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b7c036030 con 0x55bddee1aa70 2024-04-26T21:21:24.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b637fe640 1 -- 172.21.15.129:0/3563632557 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f5b7c030c50 con 0x55bddee1aa70 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b8d625740 10 client.25373 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.381+0000 7f5b8d625740 10 client.25373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.382+0000 7f5b8d625740 20 client.25373 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.382+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bddeea5bb0 0x55bddeea7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.382+0000 7f5b8d625740 4 client.25373 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.382+0000 7f5b8d625740 1 -- 172.21.15.129:0/3563632557 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bdded85800 con 0x55bddeea5bb0 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.382+0000 7f5b83fff640 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bddeea5bb0 0x55bddeea7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.382+0000 7f5b637fe640 10 client.25373 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.382+0000 7f5b83fff640 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bddeea5bb0 0x55bddeea7fd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc18ef3921c99d8 server_cookie=d99e1afe4f8778c7 in_seq=0 out_seq=0 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b637fe640 1 -- 172.21.15.129:0/3563632557 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55bdded85800 con 0x55bddeea5bb0 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b637fe640 10 client.25373 handle_command_reply: tid=0 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 client.25373 shutdown 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 -- 172.21.15.129:0/3563632557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 msgr2=0x55bddeea42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddeea42b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c031b50 tx=0x7f5b7c0095c0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 -- 172.21.15.129:0/3563632557 shutdown_connections 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bddeea5bb0 0x55bddeea7fd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bddee1aa70 0x55bddeea42b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bddee1a0b0 0x55bddee27b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b637fe640 0 client.25373 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 --2- 172.21.15.129:0/3563632557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bddee18ee0 0x55bddee275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 -- 172.21.15.129:0/3563632557 >> 172.21.15.129:0/3563632557 conn(0x55bddee143d0 msgr2=0x55bddee232d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 -- 172.21.15.129:0/3563632557 shutdown_connections 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.383+0000 7f5b8d625740 1 -- 172.21.15.129:0/3563632557 wait complete. 2024-04-26T21:21:24.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.384+0000 7f5b8d625740 20 client.25373 trim_cache size 0 max 16384 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (4629 inodes in the stack)", 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout: "eeb2d312-0673-43b5-9ab4-2b9ed46fce4a": { 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "eeb2d312-0673-43b5-9ab4-2b9ed46fce4a", 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:21:24.391 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:21:24.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.384+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b6c0a3cc0 msgr2=0x7f5b6c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.384+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b6c0a3cc0 0x7f5b6c0a6180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c04c000 tx=0x7f5b7c04d000 comp rx=0 tx=0).stop 2024-04-26T21:21:24.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.384+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b84103bd0 msgr2=0x7f5b84196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.384+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b84103bd0 0x7f5b84196ec0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5b7801fb40 tx=0x7f5b7801d2e0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.385+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1771041023 shutdown_connections 2024-04-26T21:21:24.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.385+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5b6c0a3cc0 0x7f5b6c0a6180 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.385+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b84106580 0x7f5b8419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.385+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b84103bd0 0x7f5b84196ec0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.385+0000 7f5b8a90a640 1 --2- 172.21.15.129:0/1771041023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b84101260 0x7f5b84196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.385+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1771041023 >> 172.21.15.129:0/1771041023 conn(0x7f5b840fce20 msgr2=0x7f5b840fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.385+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1771041023 shutdown_connections 2024-04-26T21:21:24.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:24.385+0000 7f5b8a90a640 1 -- 172.21.15.129:0/1771041023 wait complete. 2024-04-26T21:21:24.412 INFO:tasks.cephfs.filesystem:scrub status for tag:eeb2d312-0673-43b5-9ab4-2b9ed46fce4a - {'path': '/', 'tag': 'eeb2d312-0673-43b5-9ab4-2b9ed46fce4a', 'options': 'recursive,force'} 2024-04-26T21:21:26.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:26 smithi129 ceph-mon[29237]: pgmap v453: 97 pgs: 97 active+clean; 401 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-04-26T21:21:26.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:26 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:26 smithi171 ceph-mon[33478]: pgmap v453: 97 pgs: 97 active+clean; 401 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-04-26T21:21:26.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:26 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:26 smithi150 ceph-mon[32290]: pgmap v453: 97 pgs: 97 active+clean; 401 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-04-26T21:21:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:26 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:27 smithi171 ceph-mon[33478]: pgmap v454: 97 pgs: 97 active+clean; 401 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 115 op/s 2024-04-26T21:21:27.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:27 smithi150 ceph-mon[32290]: pgmap v454: 97 pgs: 97 active+clean; 401 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 115 op/s 2024-04-26T21:21:27.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:27 smithi129 ceph-mon[29237]: pgmap v454: 97 pgs: 97 active+clean; 401 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 115 op/s 2024-04-26T21:21:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:28.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:21:28.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:28.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:28.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:21:28.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:21:29.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:29 smithi171 ceph-mon[33478]: pgmap v455: 97 pgs: 97 active+clean; 399 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-04-26T21:21:29.886 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:29 smithi150 ceph-mon[32290]: pgmap v455: 97 pgs: 97 active+clean; 399 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-04-26T21:21:29.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:29 smithi129 ceph-mon[29237]: pgmap v455: 97 pgs: 97 active+clean; 399 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-04-26T21:21:31.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:31 smithi129 ceph-mon[29237]: pgmap v456: 97 pgs: 97 active+clean; 422 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 252 op/s 2024-04-26T21:21:31.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:31 smithi171 ceph-mon[33478]: pgmap v456: 97 pgs: 97 active+clean; 422 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 252 op/s 2024-04-26T21:21:31.768 INFO:tasks.workunit.client.0.smithi129.stdout: 89 1428795 461 962139 413 762773 1153 2024-04-26T21:21:31.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:31 smithi150 ceph-mon[32290]: pgmap v456: 97 pgs: 97 active+clean; 422 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 252 op/s 2024-04-26T21:21:33.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:33 smithi129 ceph-mon[29237]: pgmap v457: 97 pgs: 97 active+clean; 459 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 417 op/s 2024-04-26T21:21:33.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:33 smithi171 ceph-mon[33478]: pgmap v457: 97 pgs: 97 active+clean; 459 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 417 op/s 2024-04-26T21:21:33.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:33 smithi150 ceph-mon[32290]: pgmap v457: 97 pgs: 97 active+clean; 459 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 417 op/s 2024-04-26T21:21:35.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:35 smithi171 ceph-mon[33478]: pgmap v458: 97 pgs: 97 active+clean; 479 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.0 MiB/s wr, 521 op/s 2024-04-26T21:21:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:35 smithi150 ceph-mon[32290]: pgmap v458: 97 pgs: 97 active+clean; 479 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.0 MiB/s wr, 521 op/s 2024-04-26T21:21:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:35 smithi129 ceph-mon[29237]: pgmap v458: 97 pgs: 97 active+clean; 479 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.0 MiB/s wr, 521 op/s 2024-04-26T21:21:37.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:37 smithi171 ceph-mon[33478]: pgmap v459: 97 pgs: 97 active+clean; 481 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 541 op/s 2024-04-26T21:21:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:37 smithi129 ceph-mon[29237]: pgmap v459: 97 pgs: 97 active+clean; 481 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 541 op/s 2024-04-26T21:21:37.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:37 smithi150 ceph-mon[32290]: pgmap v459: 97 pgs: 97 active+clean; 481 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 541 op/s 2024-04-26T21:21:39.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:39 smithi129 ceph-mon[29237]: pgmap v460: 97 pgs: 97 active+clean; 482 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 580 op/s 2024-04-26T21:21:39.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:39 smithi171 ceph-mon[33478]: pgmap v460: 97 pgs: 97 active+clean; 482 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 580 op/s 2024-04-26T21:21:39.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:39 smithi150 ceph-mon[32290]: pgmap v460: 97 pgs: 97 active+clean; 482 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 580 op/s 2024-04-26T21:21:41.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:41 smithi171 ceph-mon[33478]: pgmap v461: 97 pgs: 97 active+clean; 479 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 597 op/s 2024-04-26T21:21:41.768 INFO:tasks.workunit.client.0.smithi129.stdout: 93 1396213 388 927665 293 753063 1237 2024-04-26T21:21:41.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:41 smithi150 ceph-mon[32290]: pgmap v461: 97 pgs: 97 active+clean; 479 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 597 op/s 2024-04-26T21:21:41.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:41 smithi129 ceph-mon[29237]: pgmap v461: 97 pgs: 97 active+clean; 479 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 597 op/s 2024-04-26T21:21:43.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:43 smithi129 ceph-mon[29237]: pgmap v462: 97 pgs: 97 active+clean; 481 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 467 op/s 2024-04-26T21:21:43.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:43 smithi171 ceph-mon[33478]: pgmap v462: 97 pgs: 97 active+clean; 481 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 467 op/s 2024-04-26T21:21:43.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:43 smithi150 ceph-mon[32290]: pgmap v462: 97 pgs: 97 active+clean; 481 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 467 op/s 2024-04-26T21:21:45.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:45 smithi171 ceph-mon[33478]: pgmap v463: 97 pgs: 97 active+clean; 480 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-04-26T21:21:45.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:45 smithi150 ceph-mon[32290]: pgmap v463: 97 pgs: 97 active+clean; 480 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-04-26T21:21:45.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:45 smithi129 ceph-mon[29237]: pgmap v463: 97 pgs: 97 active+clean; 480 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-04-26T21:21:47.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:47 smithi171 ceph-mon[33478]: pgmap v464: 97 pgs: 97 active+clean; 487 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-04-26T21:21:47.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:47 smithi150 ceph-mon[32290]: pgmap v464: 97 pgs: 97 active+clean; 487 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-04-26T21:21:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:47 smithi129 ceph-mon[29237]: pgmap v464: 97 pgs: 97 active+clean; 487 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-04-26T21:21:49.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:49 smithi171 ceph-mon[33478]: pgmap v465: 97 pgs: 97 active+clean; 485 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-04-26T21:21:49.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:49 smithi150 ceph-mon[32290]: pgmap v465: 97 pgs: 97 active+clean; 485 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-04-26T21:21:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:49 smithi129 ceph-mon[29237]: pgmap v465: 97 pgs: 97 active+clean; 485 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-04-26T21:21:51.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:51 smithi171 ceph-mon[33478]: pgmap v466: 97 pgs: 97 active+clean; 505 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-04-26T21:21:51.768 INFO:tasks.workunit.client.0.smithi129.stdout: 99 1780280 428 1210482 393 1020920 1225 2024-04-26T21:21:51.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:51 smithi150 ceph-mon[32290]: pgmap v466: 97 pgs: 97 active+clean; 505 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-04-26T21:21:51.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:51 smithi129 ceph-mon[29237]: pgmap v466: 97 pgs: 97 active+clean; 505 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-04-26T21:21:53.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:53 smithi171 ceph-mon[33478]: pgmap v467: 97 pgs: 97 active+clean; 529 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 337 op/s 2024-04-26T21:21:53.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:53 smithi150 ceph-mon[32290]: pgmap v467: 97 pgs: 97 active+clean; 529 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 337 op/s 2024-04-26T21:21:53.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:53 smithi129 ceph-mon[29237]: pgmap v467: 97 pgs: 97 active+clean; 529 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 337 op/s 2024-04-26T21:21:54.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:21:54.414 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:21:54.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:54 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:21:54.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:54 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:21:54.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:54 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:21:55.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:55 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:21:55.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:55 smithi171 ceph-mon[33478]: pgmap v468: 97 pgs: 97 active+clean; 545 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 424 op/s 2024-04-26T21:21:55.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:55 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:21:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:55 smithi150 ceph-mon[32290]: pgmap v468: 97 pgs: 97 active+clean; 545 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 424 op/s 2024-04-26T21:21:55.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:55 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:21:55.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:55 smithi129 ceph-mon[29237]: pgmap v468: 97 pgs: 97 active+clean; 545 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 424 op/s 2024-04-26T21:21:56.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.478+0000 7f6d564c2640 1 -- 172.21.15.129:0/943269542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a9b0 msgr2=0x7f6d5010f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.479+0000 7f6d564c2640 1 --2- 172.21.15.129:0/943269542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a9b0 0x7f6d5010f270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009ca0 tx=0x7f6d4802a8f0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.484+0000 7f6d564c2640 1 -- 172.21.15.129:0/943269542 shutdown_connections 2024-04-26T21:21:56.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.484+0000 7f6d564c2640 1 --2- 172.21.15.129:0/943269542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a9b0 0x7f6d5010f270 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.484+0000 7f6d564c2640 1 --2- 172.21.15.129:0/943269542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d5010a010 0x7f6d5010a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.484+0000 7f6d564c2640 1 --2- 172.21.15.129:0/943269542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d50073d70 0x7f6d50074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.484+0000 7f6d564c2640 1 -- 172.21.15.129:0/943269542 >> 172.21.15.129:0/943269542 conn(0x7f6d5006f840 msgr2=0x7f6d50071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.484+0000 7f6d564c2640 1 -- 172.21.15.129:0/943269542 shutdown_connections 2024-04-26T21:21:56.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.484+0000 7f6d564c2640 1 -- 172.21.15.129:0/943269542 wait complete. 2024-04-26T21:21:56.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.485+0000 7f6d564c2640 1 Processor -- start 2024-04-26T21:21:56.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.485+0000 7f6d564c2640 1 -- start start 2024-04-26T21:21:56.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.485+0000 7f6d564c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d50073d70 0x7f6d50116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.485+0000 7f6d564c2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a010 0x7f6d50116570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.485+0000 7f6d564c2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d5010a9b0 0x7f6d501b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.485+0000 7f6d564c2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5011d1d0 con 0x7f6d50073d70 2024-04-26T21:21:56.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.485+0000 7f6d564c2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5011d050 con 0x7f6d5010a010 2024-04-26T21:21:56.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.485+0000 7f6d564c2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5011d350 con 0x7f6d5010a9b0 2024-04-26T21:21:56.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d54cbf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a010 0x7f6d50116570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d54cbf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a010 0x7f6d50116570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60212/0 (socket says 172.21.15.129:60212) 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d54cbf640 1 -- 172.21.15.129:0/3529643232 learned_addr learned my addr 172.21.15.129:0/3529643232 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d554c0640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d50073d70 0x7f6d50116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d5010a9b0 0x7f6d501b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d55cc1640 1 -- 172.21.15.129:0/3529643232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a010 msgr2=0x7f6d50116570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a010 0x7f6d50116570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d55cc1640 1 -- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d50073d70 msgr2=0x7f6d50116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d50073d70 0x7f6d50116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d55cc1640 1 -- 172.21.15.129:0/3529643232 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d501b7e20 con 0x7f6d5010a9b0 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d554c0640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d50073d70 0x7f6d50116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d54cbf640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a010 0x7f6d50116570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:56.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.486+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d5010a9b0 0x7f6d501b78e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009bf0 tx=0x7f6d48003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.493+0000 7f6d467fc640 1 -- 172.21.15.129:0/3529643232 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4802f3b0 con 0x7f6d5010a9b0 2024-04-26T21:21:56.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.493+0000 7f6d564c2640 1 -- 172.21.15.129:0/3529643232 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d501b7ff0 con 0x7f6d5010a9b0 2024-04-26T21:21:56.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.493+0000 7f6d564c2640 1 -- 172.21.15.129:0/3529643232 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d501b8470 con 0x7f6d5010a9b0 2024-04-26T21:21:56.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.494+0000 7f6d467fc640 1 -- 172.21.15.129:0/3529643232 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d4802d070 con 0x7f6d5010a9b0 2024-04-26T21:21:56.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.494+0000 7f6d467fc640 1 -- 172.21.15.129:0/3529643232 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4802f930 con 0x7f6d5010a9b0 2024-04-26T21:21:56.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.498+0000 7f6d467fc640 1 -- 172.21.15.129:0/3529643232 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6d4802fbd0 con 0x7f6d5010a9b0 2024-04-26T21:21:56.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.499+0000 7f6d467fc640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6d3c0a3dc0 0x7f6d3c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d467fc640 1 -- 172.21.15.129:0/3529643232 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6d480f27c0 con 0x7f6d5010a9b0 2024-04-26T21:21:56.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d5a43f740 1 Processor -- start 2024-04-26T21:21:56.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d5a43f740 1 -- start start 2024-04-26T21:21:56.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d554c0640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6d3c0a3dc0 0x7f6d3c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 0x55fe90f304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31280 0x55fe90f31700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fe90e94d80 con 0x55fe90f31280 2024-04-26T21:21:56.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fe90e4e8b0 con 0x55fe90f31c40 2024-04-26T21:21:56.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.500+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fe90e90d40 con 0x55fe90f300b0 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d55cc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d55cc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f384d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60218/0 (socket says 172.21.15.129:60218) 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d55cc1640 1 -- 172.21.15.129:0/3268958275 learned_addr learned my addr 172.21.15.129:0/3268958275 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d54cbf640 1 --2- 172.21.15.129:0/3268958275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31280 0x55fe90f31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d55cc1640 1 -- 172.21.15.129:0/3268958275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 msgr2=0x55fe90f304d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/3268958275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 0x55fe90f304d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d55cc1640 1 -- 172.21.15.129:0/3268958275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31280 msgr2=0x55fe90f31700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/3268958275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31280 0x55fe90f31700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d55cc1640 1 -- 172.21.15.129:0/3268958275 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe90e1fa40 con 0x55fe90f31c40 2024-04-26T21:21:56.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d54cbf640 1 --2- 172.21.15.129:0/3268958275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31280 0x55fe90f31700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:56.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/3268958275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f384d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d480049d0 tx=0x7f6d480bf140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4992ea347251e2be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.501+0000 7f6d554c0640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6d3c0a3dc0 0x7f6d3c0a6280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009fc0 tx=0x7f6d4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.545+0000 7f6d3b7fe640 1 -- 172.21.15.129:0/3268958275 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480c2df0 con 0x55fe90f31c40 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.548+0000 7f6d3b7fe640 1 -- 172.21.15.129:0/3268958275 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d48036070 con 0x55fe90f31c40 2024-04-26T21:21:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.549+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3268958275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 msgr2=0x55fe90f384d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.549+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3268958275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f384d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d480049d0 tx=0x7f6d480bf140 comp rx=0 tx=0).stop 2024-04-26T21:21:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.549+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3268958275 shutdown_connections 2024-04-26T21:21:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.549+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3268958275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f384d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.549+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3268958275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31280 0x55fe90f31700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.549+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3268958275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 0x55fe90f304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.549+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3268958275 >> 172.21.15.129:0/3268958275 conn(0x55fe90f2b5a0 msgr2=0x55fe90f2b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.550+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3268958275 shutdown_connections 2024-04-26T21:21:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.550+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3268958275 wait complete. 2024-04-26T21:21:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.550+0000 7f6d5a43f740 1 -- start start 2024-04-26T21:21:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.550+0000 7f6d5a43f740 1 Processor -- start 2024-04-26T21:21:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f300b0 0x55fe90f3a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 0x55fe90f3aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90f41ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fe90f38a10 con 0x55fe90f31c40 2024-04-26T21:21:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fe90e1bd10 con 0x55fe90f300b0 2024-04-26T21:21:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fe90f38b90 con 0x55fe90f31280 2024-04-26T21:21:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d5a43f740 10 client.0 _ll_register_callbacks cb 0x55fe90e48210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d55cc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90f41ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d55cc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90f41ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39774/0 (socket says 172.21.15.129:39774) 2024-04-26T21:21:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d55cc1640 1 -- 172.21.15.129:0/1131639170 learned_addr learned my addr 172.21.15.129:0/1131639170 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.551+0000 7f6d54cbf640 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 0x55fe90f3aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.552+0000 7f6d554c0640 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f300b0 0x55fe90f3a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.552+0000 7f6d55cc1640 1 -- 172.21.15.129:0/1131639170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 msgr2=0x55fe90f3aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.552+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 0x55fe90f3aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.552+0000 7f6d55cc1640 1 -- 172.21.15.129:0/1131639170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f300b0 msgr2=0x55fe90f3a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.552+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f300b0 0x55fe90f3a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.552+0000 7f6d55cc1640 1 -- 172.21.15.129:0/1131639170 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe90c707d0 con 0x55fe90f31c40 2024-04-26T21:21:56.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.552+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90f41ab0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6d480bec00 tx=0x7f6d480f05e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599c7771785afdd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.571+0000 7f6d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:21:56.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.571+0000 7f6d38ff9640 1 -- 172.21.15.129:0/1131639170 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480c2df0 con 0x55fe90f31c40 2024-04-26T21:21:56.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.571+0000 7f6d38ff9640 1 -- 172.21.15.129:0/1131639170 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6d480bde90 con 0x55fe90f31c40 2024-04-26T21:21:56.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.571+0000 7f6d5a43f740 1 -- 172.21.15.129:0/1131639170 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe90ea2990 con 0x55fe90f31c40 2024-04-26T21:21:56.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.577+0000 7f6d38ff9640 1 -- 172.21.15.129:0/1131639170 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6d480c76b0 con 0x55fe90f31c40 2024-04-26T21:21:56.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.577+0000 7f6d5a43f740 10 client.15786 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:56.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.577+0000 7f6d5a43f740 1 -- 172.21.15.129:0/1131639170 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55fe90ba63f0 con 0x55fe90f31c40 2024-04-26T21:21:56.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.577+0000 7f6d38ff9640 1 -- 172.21.15.129:0/1131639170 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fe90ea2990 con 0x55fe90f31c40 2024-04-26T21:21:56.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.577+0000 7f6d38ff9640 1 -- 172.21.15.129:0/1131639170 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f6d480bd900 con 0x55fe90f31c40 2024-04-26T21:21:56.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.577+0000 7f6d5a43f740 10 client.15786 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:56.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.577+0000 7f6d5a43f740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:56.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.577+0000 7f6d5a43f740 20 client.15786 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:56.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.577+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55fe90f45ca0 0x55fe90f48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.578+0000 7f6d5a43f740 4 client.15786 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:21:56.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.578+0000 7f6d5a43f740 1 -- 172.21.15.129:0/1131639170 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe90e9c800 con 0x55fe90f45ca0 2024-04-26T21:21:56.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.578+0000 7f6d554c0640 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55fe90f45ca0 0x55fe90f48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.579+0000 7f6d554c0640 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55fe90f45ca0 0x55fe90f48090 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6813f6bc4d093958 server_cookie=751a9a1ff690ad9d in_seq=0 out_seq=0 2024-04-26T21:21:56.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.579+0000 7f6d38ff9640 10 client.15786 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:56.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.582+0000 7f6d38ff9640 1 -- 172.21.15.129:0/1131639170 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe90e9c800 con 0x55fe90f45ca0 2024-04-26T21:21:56.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.582+0000 7f6d38ff9640 10 client.15786 handle_command_reply: tid=0 2024-04-26T21:21:56.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.582+0000 7f6d5a43f740 1 client.15786 shutdown 2024-04-26T21:21:56.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d5a43f740 1 -- 172.21.15.129:0/1131639170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 msgr2=0x55fe90f41ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90f41ab0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6d480bec00 tx=0x7f6d480f05e0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d5a43f740 1 -- 172.21.15.129:0/1131639170 shutdown_connections 2024-04-26T21:21:56.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55fe90f45ca0 0x55fe90f48090 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90f41ab0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 0x55fe90f3aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/1131639170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f300b0 0x55fe90f3a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d5a43f740 1 -- 172.21.15.129:0/1131639170 >> 172.21.15.129:0/1131639170 conn(0x55fe90f2b5a0 msgr2=0x55fe90f2a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d38ff9640 0 client.15786 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:56.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.583+0000 7f6d5a43f740 1 -- 172.21.15.129:0/1131639170 shutdown_connections 2024-04-26T21:21:56.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.584+0000 7f6d5a43f740 1 -- 172.21.15.129:0/1131639170 wait complete. 2024-04-26T21:21:56.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.584+0000 7f6d5a43f740 20 client.15786 trim_cache size 0 max 16384 2024-04-26T21:21:56.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.679+0000 7f6d5a43f740 1 Processor -- start 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.679+0000 7f6d5a43f740 1 -- start start 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.679+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f300b0 0x55fe90f39f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.679+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 0x55fe90f41a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.679+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f29e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.679+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3400c4e0 con 0x55fe90f300b0 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.679+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3400ed60 con 0x55fe90f31c40 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.679+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3400cb50 con 0x55fe90f31280 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d54cbf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 0x55fe90f41a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d54cbf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 0x55fe90f41a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34180/0 (socket says 172.21.15.129:34180) 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d54cbf640 1 -- 172.21.15.129:0/740262171 learned_addr learned my addr 172.21.15.129:0/740262171 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d554c0640 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f300b0 0x55fe90f39f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f29e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d55cc1640 1 -- 172.21.15.129:0/740262171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 msgr2=0x55fe90f41a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 0x55fe90f41a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d55cc1640 1 -- 172.21.15.129:0/740262171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f300b0 msgr2=0x55fe90f39f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f300b0 0x55fe90f39f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d55cc1640 1 -- 172.21.15.129:0/740262171 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe90b660f0 con 0x55fe90f31c40 2024-04-26T21:21:56.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.680+0000 7f6d554c0640 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f300b0 0x55fe90f39f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:56.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.681+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f29e20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d480bf2f0 tx=0x7f6d480bf140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4249b3db605858e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.693+0000 7f6d3affd640 1 -- 172.21.15.129:0/740262171 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480c2df0 con 0x55fe90f31c40 2024-04-26T21:21:56.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.693+0000 7f6d3affd640 1 -- 172.21.15.129:0/740262171 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d48036070 con 0x55fe90f31c40 2024-04-26T21:21:56.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.694+0000 7f6d5a43f740 1 -- 172.21.15.129:0/740262171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 msgr2=0x55fe90f29e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.694+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f29e20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d480bf2f0 tx=0x7f6d480bf140 comp rx=0 tx=0).stop 2024-04-26T21:21:56.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.697+0000 7f6d5a43f740 1 -- 172.21.15.129:0/740262171 shutdown_connections 2024-04-26T21:21:56.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.697+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31c40 0x55fe90f29e20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.697+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f31280 0x55fe90f41a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.697+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/740262171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f300b0 0x55fe90f39f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.697+0000 7f6d5a43f740 1 -- 172.21.15.129:0/740262171 >> 172.21.15.129:0/740262171 conn(0x55fe90f2b5a0 msgr2=0x55fe90f39800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.697+0000 7f6d5a43f740 1 -- 172.21.15.129:0/740262171 shutdown_connections 2024-04-26T21:21:56.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.697+0000 7f6d5a43f740 1 -- 172.21.15.129:0/740262171 wait complete. 2024-04-26T21:21:56.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.698+0000 7f6d5a43f740 1 -- start start 2024-04-26T21:21:56.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.698+0000 7f6d5a43f740 1 Processor -- start 2024-04-26T21:21:56.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 0x55fe90f3c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31280 0x55fe90f3cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d5a43f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90faa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3400d220 con 0x55fe90f31c40 2024-04-26T21:21:56.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3400e690 con 0x55fe90f31280 2024-04-26T21:21:56.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d5a43f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3400d8f0 con 0x55fe90f300b0 2024-04-26T21:21:56.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d5a43f740 10 client.0 _ll_register_callbacks cb 0x7f6d480be670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:56.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d55cc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90faa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d54cbf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31280 0x55fe90f3cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d554c0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 0x55fe90f3c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d54cbf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31280 0x55fe90f3cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60242/0 (socket says 172.21.15.129:60242) 2024-04-26T21:21:56.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.699+0000 7f6d54cbf640 1 -- 172.21.15.129:0/3160332423 learned_addr learned my addr 172.21.15.129:0/3160332423 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.700+0000 7f6d54cbf640 1 -- 172.21.15.129:0/3160332423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 msgr2=0x55fe90f3c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.700+0000 7f6d54cbf640 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 0x55fe90f3c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.700+0000 7f6d54cbf640 1 -- 172.21.15.129:0/3160332423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 msgr2=0x55fe90faa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.700+0000 7f6d54cbf640 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90faa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.700+0000 7f6d54cbf640 1 -- 172.21.15.129:0/3160332423 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe90e554b0 con 0x55fe90f31280 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.700+0000 7f6d554c0640 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 0x55fe90f3c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.700+0000 7f6d55cc1640 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90faa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.700+0000 7f6d54cbf640 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31280 0x55fe90f3cb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d40017c70 tx=0x7f6d40015790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=620824e2013bfc21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.701+0000 7f6d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.701+0000 7f6d38ff9640 1 -- 172.21.15.129:0/3160332423 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40020640 con 0x55fe90f31280 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.701+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3160332423 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d48036070 con 0x55fe90f31280 2024-04-26T21:21:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.701+0000 7f6d38ff9640 1 -- 172.21.15.129:0/3160332423 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6d4001f3e0 con 0x55fe90f31280 2024-04-26T21:21:56.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.701+0000 7f6d38ff9640 1 -- 172.21.15.129:0/3160332423 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6d40009e10 con 0x55fe90f31280 2024-04-26T21:21:56.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.701+0000 7f6d5a43f740 10 client.25391 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:56.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.701+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3160332423 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55fe90fab3e0 con 0x55fe90f31280 2024-04-26T21:21:56.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.702+0000 7f6d38ff9640 1 -- 172.21.15.129:0/3160332423 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d48036070 con 0x55fe90f31280 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.702+0000 7f6d38ff9640 1 -- 172.21.15.129:0/3160332423 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f6d40029050 con 0x55fe90f31280 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.702+0000 7f6d5a43f740 10 client.25391 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.702+0000 7f6d5a43f740 10 client.25391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.702+0000 7f6d5a43f740 20 client.25391 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.702+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55fe90fabdf0 0x55fe90fae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.702+0000 7f6d5a43f740 4 client.25391 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.702+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3160332423 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe90e174a0 con 0x55fe90fabdf0 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.702+0000 7f6d554c0640 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55fe90fabdf0 0x55fe90fae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.703+0000 7f6d38ff9640 10 client.25391 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:56.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.703+0000 7f6d554c0640 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55fe90fabdf0 0x55fe90fae210 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b924a10d43034181 server_cookie=5a415301045ce477 in_seq=0 out_seq=0 2024-04-26T21:21:56.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.709+0000 7f6d38ff9640 1 -- 172.21.15.129:0/3160332423 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fe90e174a0 con 0x55fe90fabdf0 2024-04-26T21:21:56.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.709+0000 7f6d38ff9640 10 client.25391 handle_command_reply: tid=0 2024-04-26T21:21:56.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.709+0000 7f6d5a43f740 1 client.25391 shutdown 2024-04-26T21:21:56.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.712+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3160332423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31280 msgr2=0x55fe90f3cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.712+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31280 0x55fe90f3cb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d40017c70 tx=0x7f6d40015790 comp rx=0 tx=0).stop 2024-04-26T21:21:56.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.712+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3160332423 shutdown_connections 2024-04-26T21:21:56.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.712+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55fe90fabdf0 0x55fe90fae210 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.712+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe90f31280 0x55fe90f3cb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.712+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe90f31c40 0x55fe90faa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.712+0000 7f6d5a43f740 1 --2- 172.21.15.129:0/3160332423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe90f300b0 0x55fe90f3c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.712+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3160332423 >> 172.21.15.129:0/3160332423 conn(0x55fe90f2b5a0 msgr2=0x55fe90f2e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.712+0000 7f6d38ff9640 0 client.25391 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:56.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.713+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3160332423 shutdown_connections 2024-04-26T21:21:56.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.713+0000 7f6d5a43f740 1 -- 172.21.15.129:0/3160332423 wait complete. 2024-04-26T21:21:56.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.713+0000 7f6d5a43f740 20 client.25391 trim_cache size 0 max 16384 2024-04-26T21:21:56.715 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:21:56.715 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:21:56.715 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:21:56.715 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:21:56.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.723+0000 7f6d564c2640 1 -- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6d3c0a3dc0 msgr2=0x7f6d3c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.723+0000 7f6d564c2640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6d3c0a3dc0 0x7f6d3c0a6280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009fc0 tx=0x7f6d4c005f90 comp rx=0 tx=0).stop 2024-04-26T21:21:56.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.723+0000 7f6d564c2640 1 -- 172.21.15.129:0/3529643232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d5010a9b0 msgr2=0x7f6d501b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.723+0000 7f6d564c2640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d5010a9b0 0x7f6d501b78e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009bf0 tx=0x7f6d48003ba0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.723+0000 7f6d564c2640 1 -- 172.21.15.129:0/3529643232 shutdown_connections 2024-04-26T21:21:56.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.724+0000 7f6d564c2640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f6d3c0a3dc0 0x7f6d3c0a6280 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.724+0000 7f6d564c2640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5010a010 0x7f6d50116570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.724+0000 7f6d564c2640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d5010a9b0 0x7f6d501b78e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.724+0000 7f6d564c2640 1 --2- 172.21.15.129:0/3529643232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d50073d70 0x7f6d50116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.724+0000 7f6d564c2640 1 -- 172.21.15.129:0/3529643232 >> 172.21.15.129:0/3529643232 conn(0x7f6d5006f840 msgr2=0x7f6d500714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.724+0000 7f6d564c2640 1 -- 172.21.15.129:0/3529643232 shutdown_connections 2024-04-26T21:21:56.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.724+0000 7f6d564c2640 1 -- 172.21.15.129:0/3529643232 wait complete. 2024-04-26T21:21:56.747 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:21:56.747 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:56.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.888+0000 7fdb9f33c640 1 -- 172.21.15.129:0/3410223305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90098020 msgr2=0x7fdb90098440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.888+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/3410223305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90098020 0x7fdb90098440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009900 tx=0x7fdb9402a410 comp rx=0 tx=0).stop 2024-04-26T21:21:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.889+0000 7fdb9f33c640 1 -- 172.21.15.129:0/3410223305 shutdown_connections 2024-04-26T21:21:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.889+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/3410223305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb90099bd0 0x7fdb900a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.889+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/3410223305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb90099210 0x7fdb90099690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.889+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/3410223305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90098020 0x7fdb90098440 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.889+0000 7fdb9f33c640 1 -- 172.21.15.129:0/3410223305 >> 172.21.15.129:0/3410223305 conn(0x7fdb900935b0 msgr2=0x7fdb900959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.889+0000 7fdb9f33c640 1 -- 172.21.15.129:0/3410223305 shutdown_connections 2024-04-26T21:21:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.889+0000 7fdb9f33c640 1 -- 172.21.15.129:0/3410223305 wait complete. 2024-04-26T21:21:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.890+0000 7fdb9f33c640 1 Processor -- start 2024-04-26T21:21:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.891+0000 7fdb9f33c640 1 -- start start 2024-04-26T21:21:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.891+0000 7fdb9f33c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb90098020 0x7fdb9012e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.891+0000 7fdb9f33c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb90099210 0x7fdb9012e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.891+0000 7fdb9f33c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90099bd0 0x7fdb90135600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.891+0000 7fdb9e33a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb90098020 0x7fdb9012e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.891+0000 7fdb9e33a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb90098020 0x7fdb9012e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39810/0 (socket says 172.21.15.129:39810) 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.891+0000 7fdb9db39640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb90099210 0x7fdb9012e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.891+0000 7fdb9db39640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb90099210 0x7fdb9012e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34216/0 (socket says 172.21.15.129:34216) 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9eb3b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90099bd0 0x7fdb90135600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9eb3b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90099bd0 0x7fdb90135600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60250/0 (socket says 172.21.15.129:60250) 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9e33a640 1 -- 172.21.15.129:0/1187690657 learned_addr learned my addr 172.21.15.129:0/1187690657 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb900a3bb0 con 0x7fdb90098020 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdb900a3a30 con 0x7fdb90099bd0 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdb900a3d30 con 0x7fdb90099210 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9e33a640 1 -- 172.21.15.129:0/1187690657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb90099210 msgr2=0x7fdb9012e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9e33a640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb90099210 0x7fdb9012e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9e33a640 1 -- 172.21.15.129:0/1187690657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90099bd0 msgr2=0x7fdb90135600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9e33a640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90099bd0 0x7fdb90135600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9e33a640 1 -- 172.21.15.129:0/1187690657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb90135c70 con 0x7fdb90098020 2024-04-26T21:21:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.892+0000 7fdb9eb3b640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90099bd0 0x7fdb90135600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.893+0000 7fdb9e33a640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb90098020 0x7fdb9012e050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009230 tx=0x7fdb940052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.893+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1187690657 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94004230 con 0x7fdb90098020 2024-04-26T21:21:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.893+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1187690657 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb94030e70 con 0x7fdb90098020 2024-04-26T21:21:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.893+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1187690657 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94004af0 con 0x7fdb90098020 2024-04-26T21:21:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.893+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb90135f00 con 0x7fdb90098020 2024-04-26T21:21:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.893+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb901362b0 con 0x7fdb90098020 2024-04-26T21:21:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.896+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb901364e0 con 0x7fdb90098020 2024-04-26T21:21:56.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.897+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1187690657 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdb94004cf0 con 0x7fdb90098020 2024-04-26T21:21:56.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.898+0000 7fdb8f7fe640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdb840a3cc0 0x7fdb840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.898+0000 7fdb9db39640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdb840a3cc0 0x7fdb840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.899+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1187690657 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fdb940f1350 con 0x7fdb90098020 2024-04-26T21:21:56.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.899+0000 7fdb9db39640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdb840a3cc0 0x7fdb840a6180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb9012f560 tx=0x7fdb8800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:56.907+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1187690657 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdb940bd570 con 0x7fdb90098020 2024-04-26T21:21:57.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:57 smithi171 ceph-mon[33478]: pgmap v469: 97 pgs: 97 active+clean; 547 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 456 op/s 2024-04-26T21:21:57.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:57 smithi150 ceph-mon[32290]: pgmap v469: 97 pgs: 97 active+clean; 547 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 456 op/s 2024-04-26T21:21:57.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:57 smithi129 ceph-mon[29237]: pgmap v469: 97 pgs: 97 active+clean; 547 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 456 op/s 2024-04-26T21:21:58.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.357+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb90136870 con 0x7fdb90098020 2024-04-26T21:21:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.359+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1187690657 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7fdb940c1870 con 0x7fdb90098020 2024-04-26T21:21:58.360 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:21:58.360 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:21:58.360 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:21:58.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.363+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdb840a3cc0 msgr2=0x7fdb840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.363+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdb840a3cc0 0x7fdb840a6180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb9012f560 tx=0x7fdb8800b390 comp rx=0 tx=0).stop 2024-04-26T21:21:58.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.363+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb90098020 msgr2=0x7fdb9012e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.363+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb90098020 0x7fdb9012e050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009230 tx=0x7fdb940052f0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.363+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 shutdown_connections 2024-04-26T21:21:58.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.363+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdb840a3cc0 0x7fdb840a6180 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.364+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb90099bd0 0x7fdb90135600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.364+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb90099210 0x7fdb9012e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.364+0000 7fdb9f33c640 1 --2- 172.21.15.129:0/1187690657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb90098020 0x7fdb9012e050 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.364+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 >> 172.21.15.129:0/1187690657 conn(0x7fdb900935b0 msgr2=0x7fdb9009c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:58.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.364+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 shutdown_connections 2024-04-26T21:21:58.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.364+0000 7fdb9f33c640 1 -- 172.21.15.129:0/1187690657 wait complete. 2024-04-26T21:21:58.387 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:21:58.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/1993879066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 msgr2=0x7f8bf4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/1993879066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 0x7f8bf4102da0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009900 tx=0x7f8bf002a410 comp rx=0 tx=0).stop 2024-04-26T21:21:58.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/1993879066 shutdown_connections 2024-04-26T21:21:58.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/1993879066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf4103b70 0x7f8bf4103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/1993879066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf4104530 0x7f8bf410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/1993879066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 0x7f8bf4102da0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/1993879066 >> 172.21.15.129:0/1993879066 conn(0x7f8bf40fdf10 msgr2=0x7f8bf4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/1993879066 shutdown_connections 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/1993879066 wait complete. 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 Processor -- start 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 -- start start 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 0x7f8bf4198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf4103b70 0x7f8bf4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf4104530 0x7f8bf41a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410e4b0 con 0x7f8bf4102980 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410e330 con 0x7f8bf4103b70 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.506+0000 7f8bfbcbd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410e630 con 0x7f8bf4104530 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9a59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 0x7f8bf4198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9a59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 0x7f8bf4198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39834/0 (socket says 172.21.15.129:39834) 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9a59640 1 -- 172.21.15.129:0/3230647370 learned_addr learned my addr 172.21.15.129:0/3230647370 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9258640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf4103b70 0x7f8bf4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bfa25a640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf4104530 0x7f8bf41a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9258640 1 -- 172.21.15.129:0/3230647370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf4104530 msgr2=0x7f8bf41a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9258640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf4104530 0x7f8bf41a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9258640 1 -- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 msgr2=0x7f8bf4198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9258640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 0x7f8bf4198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9258640 1 -- 172.21.15.129:0/3230647370 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf41a0840 con 0x7f8bf4103b70 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 0x7f8bf4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bf9258640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf4103b70 0x7f8bf4199170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8be400e9f0 tx=0x7f8be400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8beaffd640 1 -- 172.21.15.129:0/3230647370 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be400ccc0 con 0x7f8bf4103b70 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8beaffd640 1 -- 172.21.15.129:0/3230647370 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be40108e0 con 0x7f8bf4103b70 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/3230647370 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf41a0b30 con 0x7f8bf4103b70 2024-04-26T21:21:58.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8beaffd640 1 -- 172.21.15.129:0/3230647370 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4018880 con 0x7f8bf4103b70 2024-04-26T21:21:58.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.507+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/3230647370 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a1080 con 0x7f8bf4103b70 2024-04-26T21:21:58.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.509+0000 7f8beaffd640 1 -- 172.21.15.129:0/3230647370 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8be4010430 con 0x7f8bf4103b70 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.510+0000 7f8bfe9d5740 1 Processor -- start 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8beaffd640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8be00a3c40 0x7f8be00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bfe9d5740 1 -- start start 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 0x5555eda71280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bfa25a640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8be00a3c40 0x7f8be00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda78050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5555ed9cfd80 con 0x5555eda6fc30 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5555ed989a20 con 0x5555eda717c0 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5555ed9cbd40 con 0x5555eda70e00 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8beaffd640 1 -- 172.21.15.129:0/3230647370 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f8be40d0980 con 0x7f8bf4103b70 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bf9a59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda70050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bf9a59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda70050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39836/0 (socket says 172.21.15.129:39836) 2024-04-26T21:21:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bf9a59640 1 -- 172.21.15.129:0/1410075968 learned_addr learned my addr 172.21.15.129:0/1410075968 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bfa25a640 1 --2- 172.21.15.129:0/1410075968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda78050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bf9258640 1 --2- 172.21.15.129:0/1410075968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 0x5555eda71280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bf9a59640 1 -- 172.21.15.129:0/1410075968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 msgr2=0x5555eda71280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/1410075968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 0x5555eda71280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bf9a59640 1 -- 172.21.15.129:0/1410075968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 msgr2=0x5555eda78050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/1410075968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda78050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.511+0000 7f8bf9a59640 1 -- 172.21.15.129:0/1410075968 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ed994c30 con 0x5555eda6fc30 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.512+0000 7f8bfa25a640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8be00a3c40 0x7f8be00a6100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bec005fd0 tx=0x7f8bec014bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.512+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/1410075968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda70050 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005ea0 tx=0x7f8bf00315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e905cfe7551e08a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.512+0000 7f8bdffff640 1 -- 172.21.15.129:0/1410075968 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0040070 con 0x5555eda6fc30 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.512+0000 7f8bdffff640 1 -- 172.21.15.129:0/1410075968 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf0005040 con 0x5555eda6fc30 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.512+0000 7f8bdffff640 1 -- 172.21.15.129:0/1410075968 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0035040 con 0x5555eda6fc30 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.513+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/1410075968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 msgr2=0x5555eda70050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.513+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/1410075968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda70050 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005ea0 tx=0x7f8bf00315a0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.513+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/1410075968 shutdown_connections 2024-04-26T21:21:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.513+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/1410075968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda78050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.513+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/1410075968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 0x5555eda71280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.513+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/1410075968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda70050 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.513+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/1410075968 >> 172.21.15.129:0/1410075968 conn(0x5555ed9ac910 msgr2=0x5555eda67120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.513+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/1410075968 shutdown_connections 2024-04-26T21:21:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.513+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/1410075968 wait complete. 2024-04-26T21:21:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.514+0000 7f8bfe9d5740 1 -- start start 2024-04-26T21:21:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.514+0000 7f8bfe9d5740 1 Processor -- start 2024-04-26T21:21:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.514+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda6fc30 0x5555eda79e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.514+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda70e00 0x5555eda7a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.514+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda717c0 0x5555eda81400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.514+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5555eda78770 con 0x5555eda70e00 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5555ed956dc0 con 0x5555eda6fc30 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5555eda788f0 con 0x5555eda717c0 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bfe9d5740 10 client.0 _ll_register_callbacks cb 0x5555ed972600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9258640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda70e00 0x5555eda7a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9a59640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda6fc30 0x5555eda79e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9a59640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda6fc30 0x5555eda79e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60284/0 (socket says 172.21.15.129:60284) 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9258640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda70e00 0x5555eda7a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39852/0 (socket says 172.21.15.129:39852) 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9a59640 1 -- 172.21.15.129:0/608338462 learned_addr learned my addr 172.21.15.129:0/608338462 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bfa25a640 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda717c0 0x5555eda81400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9258640 1 -- 172.21.15.129:0/608338462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda717c0 msgr2=0x5555eda81400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9258640 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda717c0 0x5555eda81400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9258640 1 -- 172.21.15.129:0/608338462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda6fc30 msgr2=0x5555eda79e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:58.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9258640 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda6fc30 0x5555eda79e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9258640 1 -- 172.21.15.129:0/608338462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ed6e13f0 con 0x5555eda70e00 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda6fc30 0x5555eda79e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bf9258640 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda70e00 0x5555eda7a3c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8be40ce2c0 tx=0x7f8be4010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef1827727934ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/608338462 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40a5700 con 0x5555eda70e00 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/608338462 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f8be4010a90 con 0x5555eda70e00 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/608338462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bf0005040 con 0x5555eda70e00 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.515+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/608338462 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8be409bf90 con 0x5555eda70e00 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bfe9d5740 10 client.15810 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/608338462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5555ed7ab7d0 con 0x5555eda70e00 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/608338462 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf0005040 con 0x5555eda70e00 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/608338462 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f8be409bc00 con 0x5555eda70e00 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bfe9d5740 10 client.15810 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:58.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bfe9d5740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bfe9d5740 20 client.15810 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5555eda85ce0 0x5555eda88150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bfe9d5740 4 client.15810 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:21:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/608338462 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ed9dd990 con 0x5555eda85ce0 2024-04-26T21:21:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.516+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5555eda85ce0 0x5555eda88150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.517+0000 7f8bdd7fa640 10 client.15810 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.517+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5555eda85ce0 0x5555eda88150 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3daa28a665fdb62e server_cookie=a07ce30e4c4c7ddb in_seq=0 out_seq=0 2024-04-26T21:21:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/608338462 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555ed9dd990 con 0x5555eda85ce0 2024-04-26T21:21:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bdd7fa640 10 client.15810 handle_command_reply: tid=0 2024-04-26T21:21:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 client.15810 shutdown 2024-04-26T21:21:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/608338462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda70e00 msgr2=0x5555eda7a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda70e00 0x5555eda7a3c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8be40ce2c0 tx=0x7f8be4010720 comp rx=0 tx=0).stop 2024-04-26T21:21:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/608338462 shutdown_connections 2024-04-26T21:21:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5555eda85ce0 0x5555eda88150 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda717c0 0x5555eda81400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda70e00 0x5555eda7a3c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/608338462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda6fc30 0x5555eda79e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bdd7fa640 0 client.15810 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:58.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/608338462 >> 172.21.15.129:0/608338462 conn(0x5555ed9ac910 msgr2=0x5555eda66280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:58.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/608338462 shutdown_connections 2024-04-26T21:21:58.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.521+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/608338462 wait complete. 2024-04-26T21:21:58.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.522+0000 7f8bfe9d5740 20 client.15810 trim_cache size 0 max 16384 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.535+0000 7f8bfe9d5740 1 Processor -- start 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.535+0000 7f8bfe9d5740 1 -- start start 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 0x5555eda813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda65f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41a5bc0 con 0x5555eda6fc30 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41a8440 con 0x5555eda717c0 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41a6230 con 0x5555eda70e00 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfa25a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda65f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bf9a59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfa25a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda65f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60290/0 (socket says 172.21.15.129:60290) 2024-04-26T21:21:58.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfa25a640 1 -- 172.21.15.129:0/58170517 learned_addr learned my addr 172.21.15.129:0/58170517 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bf9258640 1 --2- 172.21.15.129:0/58170517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 0x5555eda813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfa25a640 1 -- 172.21.15.129:0/58170517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 msgr2=0x5555eda813c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfa25a640 1 --2- 172.21.15.129:0/58170517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 0x5555eda813c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfa25a640 1 -- 172.21.15.129:0/58170517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 msgr2=0x5555eda659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfa25a640 1 --2- 172.21.15.129:0/58170517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bfa25a640 1 -- 172.21.15.129:0/58170517 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555eda70880 con 0x5555eda717c0 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.536+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/58170517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.537+0000 7f8bfa25a640 1 --2- 172.21.15.129:0/58170517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda65f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bec012c80 tx=0x7f8bec01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71857bca9ef024b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.537+0000 7f8bdf7fe640 1 -- 172.21.15.129:0/58170517 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec027070 con 0x5555eda717c0 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.537+0000 7f8bdf7fe640 1 -- 172.21.15.129:0/58170517 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bec015e50 con 0x5555eda717c0 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.537+0000 7f8bdf7fe640 1 -- 172.21.15.129:0/58170517 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec02b660 con 0x5555eda717c0 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/58170517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 msgr2=0x5555eda65f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/58170517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda65f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bec012c80 tx=0x7f8bec01c000 comp rx=0 tx=0).stop 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/58170517 shutdown_connections 2024-04-26T21:21:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/58170517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda717c0 0x5555eda65f30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/58170517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda70e00 0x5555eda813c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/58170517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda6fc30 0x5555eda659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/58170517 >> 172.21.15.129:0/58170517 conn(0x5555ed9ac910 msgr2=0x5555eda746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/58170517 shutdown_connections 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/58170517 wait complete. 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 -- start start 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 Processor -- start 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda6fc30 0x5555eda7e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda70e00 0x5555eda7e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.538+0000 7f8bfe9d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda717c0 0x5555eda7f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41a6900 con 0x5555eda717c0 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41a7d70 con 0x5555eda70e00 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bfe9d5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41a6fd0 con 0x5555eda6fc30 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bfe9d5740 10 client.0 _ll_register_callbacks cb 0x7f8be4098d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9258640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda70e00 0x5555eda7e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bfa25a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda717c0 0x5555eda7f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9258640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda70e00 0x5555eda7e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60300/0 (socket says 172.21.15.129:60300) 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9258640 1 -- 172.21.15.129:0/3139642678 learned_addr learned my addr 172.21.15.129:0/3139642678 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda6fc30 0x5555eda7e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9258640 1 -- 172.21.15.129:0/3139642678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda6fc30 msgr2=0x5555eda7e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9258640 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda6fc30 0x5555eda7e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9258640 1 -- 172.21.15.129:0/3139642678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda717c0 msgr2=0x5555eda7f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9258640 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda717c0 0x5555eda7f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9258640 1 -- 172.21.15.129:0/3139642678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555eda7ff90 con 0x5555eda70e00 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bfa25a640 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda717c0 0x5555eda7f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bf9258640 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda70e00 0x5555eda7e990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8be4099430 tx=0x7f8be4009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c52d74a4c1d73fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/3139642678 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be407f070 con 0x5555eda70e00 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/3139642678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bec015e50 con 0x5555eda70e00 2024-04-26T21:21:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.539+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/3139642678 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f8be4098a00 con 0x5555eda70e00 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/3139642678 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8be409c4a0 con 0x5555eda70e00 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bfe9d5740 10 client.25415 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/3139642678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5555eda81300 con 0x5555eda70e00 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/3139642678 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bec015e50 con 0x5555eda70e00 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/3139642678 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f8be4082030 con 0x5555eda70e00 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bfe9d5740 10 client.25415 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bfe9d5740 10 client.25415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bfe9d5740 20 client.25415 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5555edaec6a0 0x5555edaeeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bfe9d5740 4 client.25415 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/3139642678 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555ed9d7800 con 0x5555edaec6a0 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.540+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5555edaec6a0 0x5555edaeeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.541+0000 7f8bdd7fa640 10 client.25415 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.541+0000 7f8bf9a59640 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5555edaec6a0 0x5555edaeeac0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab69dc40b8639e4 server_cookie=fd8e87cd3ab54020 in_seq=0 out_seq=0 2024-04-26T21:21:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.546+0000 7f8bdd7fa640 1 -- 172.21.15.129:0/3139642678 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555ed9d7800 con 0x5555edaec6a0 2024-04-26T21:21:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.546+0000 7f8bdd7fa640 10 client.25415 handle_command_reply: tid=0 2024-04-26T21:21:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.546+0000 7f8bfe9d5740 1 client.25415 shutdown 2024-04-26T21:21:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.546+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/3139642678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda70e00 msgr2=0x5555eda7e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.546+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda70e00 0x5555eda7e990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8be4099430 tx=0x7f8be4009700 comp rx=0 tx=0).stop 2024-04-26T21:21:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/3139642678 shutdown_connections 2024-04-26T21:21:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5555edaec6a0 0x5555edaeeac0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555eda70e00 0x5555eda7e990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555eda717c0 0x5555eda7f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bdd7fa640 0 client.25415 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfe9d5740 1 --2- 172.21.15.129:0/3139642678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555eda6fc30 0x5555eda7e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/3139642678 >> 172.21.15.129:0/3139642678 conn(0x5555ed9ac910 msgr2=0x5555eda6a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/3139642678 shutdown_connections 2024-04-26T21:21:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfe9d5740 1 -- 172.21.15.129:0/3139642678 wait complete. 2024-04-26T21:21:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfe9d5740 20 client.25415 trim_cache size 0 max 16384 2024-04-26T21:21:58.549 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:21:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8be00a3c40 msgr2=0x7f8be00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8be00a3c40 0x7f8be00a6100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bec005fd0 tx=0x7f8bec014bf0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/3230647370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf4103b70 msgr2=0x7f8bf4199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf4103b70 0x7f8bf4199170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8be400e9f0 tx=0x7f8be400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/3230647370 shutdown_connections 2024-04-26T21:21:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8be00a3c40 0x7f8be00a6100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf4103b70 0x7f8bf4199170 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf4104530 0x7f8bf41a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 --2- 172.21.15.129:0/3230647370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8bf4102980 0x7f8bf4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:58.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.547+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/3230647370 >> 172.21.15.129:0/3230647370 conn(0x7f8bf40fdf10 msgr2=0x7f8bf4106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:58.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.548+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/3230647370 shutdown_connections 2024-04-26T21:21:58.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:58.548+0000 7f8bfbcbd640 1 -- 172.21.15.129:0/3230647370 wait complete. 2024-04-26T21:21:58.724 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:58 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1187690657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:21:58.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:58 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1187690657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:21:58.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:58 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1187690657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:21:59.562 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:21:59.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:59.563 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 -- 172.21.15.129:0/4289324849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98102a60 msgr2=0x7fdf98102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 --2- 172.21.15.129:0/4289324849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98102a60 0x7fdf98102ee0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009900 tx=0x7fdf8002a410 comp rx=0 tx=0).stop 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 -- 172.21.15.129:0/4289324849 shutdown_connections 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 --2- 172.21.15.129:0/4289324849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf98103420 0x7fdf98109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 --2- 172.21.15.129:0/4289324849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98102a60 0x7fdf98102ee0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 --2- 172.21.15.129:0/4289324849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf98101870 0x7fdf98101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 -- 172.21.15.129:0/4289324849 >> 172.21.15.129:0/4289324849 conn(0x7fdf980fcde0 msgr2=0x7fdf980ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 -- 172.21.15.129:0/4289324849 shutdown_connections 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 -- 172.21.15.129:0/4289324849 wait complete. 2024-04-26T21:21:59.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 Processor -- start 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 -- start start 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98101870 0x7fdf9806a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf98102a60 0x7fdf9819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf98103420 0x7fdf9819ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810d3a0 con 0x7fdf98101870 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810d220 con 0x7fdf98103420 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf9df78640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810d520 con 0x7fdf98102a60 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98101870 0x7fdf9806a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98101870 0x7fdf9806a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39878/0 (socket says 172.21.15.129:39878) 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf977fe640 1 -- 172.21.15.129:0/3805111344 learned_addr learned my addr 172.21.15.129:0/3805111344 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.683+0000 7fdf97fff640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf98103420 0x7fdf9819ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf8ffff640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf98102a60 0x7fdf9819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf977fe640 1 -- 172.21.15.129:0/3805111344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf98102a60 msgr2=0x7fdf9819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf977fe640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf98102a60 0x7fdf9819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf977fe640 1 -- 172.21.15.129:0/3805111344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf98103420 msgr2=0x7fdf9819ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf977fe640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf98103420 0x7fdf9819ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf977fe640 1 -- 172.21.15.129:0/3805111344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf981a0880 con 0x7fdf98101870 2024-04-26T21:21:59.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf977fe640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98101870 0x7fdf9806a630 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800e9c0 tx=0x7fdf8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf957fa640 1 -- 172.21.15.129:0/3805111344 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800cd80 con 0x7fdf98101870 2024-04-26T21:21:59.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf9df78640 1 -- 172.21.15.129:0/3805111344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf981a0b10 con 0x7fdf98101870 2024-04-26T21:21:59.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf9df78640 1 -- 172.21.15.129:0/3805111344 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf981a1080 con 0x7fdf98101870 2024-04-26T21:21:59.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf957fa640 1 -- 172.21.15.129:0/3805111344 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf880035f0 con 0x7fdf98101870 2024-04-26T21:21:59.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.684+0000 7fdf957fa640 1 -- 172.21.15.129:0/3805111344 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88010690 con 0x7fdf98101870 2024-04-26T21:21:59.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.686+0000 7fdf957fa640 1 -- 172.21.15.129:0/3805111344 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdf88003a60 con 0x7fdf98101870 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.686+0000 7fdfa0d6c740 1 Processor -- start 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.686+0000 7fdfa0d6c740 1 -- start start 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d670cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d675da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d6788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55585d5d5b00 con 0x55585d676410 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55585d571920 con 0x55585d6708b0 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55585d5d1ac0 con 0x55585d6712a0 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf957fa640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdf840a3cc0 0x7fdf840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf8ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d6788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d670cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d670cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60326/0 (socket says 172.21.15.129:60326) 2024-04-26T21:21:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 -- 172.21.15.129:0/3752860219 learned_addr learned my addr 172.21.15.129:0/3752860219 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdf840a3cc0 0x7fdf840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf957fa640 1 -- 172.21.15.129:0/3805111344 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fdf880d0890 con 0x7fdf98101870 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf977fe640 1 --2- 172.21.15.129:0/3752860219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d675da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 -- 172.21.15.129:0/3752860219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 msgr2=0x55585d675da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 --2- 172.21.15.129:0/3752860219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d675da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 -- 172.21.15.129:0/3752860219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 msgr2=0x55585d6788a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 --2- 172.21.15.129:0/3752860219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d6788a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 -- 172.21.15.129:0/3752860219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585d3af7e0 con 0x55585d6708b0 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 --2- 172.21.15.129:0/3752860219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d670cd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf90012d80 tx=0x7fdf90010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e09373705eb566de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf97fff640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdf840a3cc0 0x7fdf840a6180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdf90010c40 tx=0x7fdf9000c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.687+0000 7fdf8dffb640 1 -- 172.21.15.129:0/3752860219 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90024070 con 0x55585d6708b0 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdf8dffb640 1 -- 172.21.15.129:0/3752860219 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf900125a0 con 0x55585d6708b0 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdf8dffb640 1 -- 172.21.15.129:0/3752860219 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90005280 con 0x55585d6708b0 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3752860219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 msgr2=0x55585d670cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3752860219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d670cd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf90012d80 tx=0x7fdf90010740 comp rx=0 tx=0).stop 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3752860219 shutdown_connections 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3752860219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d6788a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3752860219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d675da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3752860219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d670cd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3752860219 >> 172.21.15.129:0/3752860219 conn(0x55585d66bf20 msgr2=0x55585d66c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3752860219 shutdown_connections 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3752860219 wait complete. 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.688+0000 7fdfa0d6c740 1 -- start start 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdfa0d6c740 1 Processor -- start 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d67b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d682220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55585d679360 con 0x55585d676410 2024-04-26T21:21:59.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55585d56f5d0 con 0x55585d6708b0 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55585d6794e0 con 0x55585d6712a0 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdfa0d6c740 10 client.0 _ll_register_callbacks cb 0x55585d55d080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf97fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d682220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf977fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf97fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d682220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39896/0 (socket says 172.21.15.129:39896) 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf977fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60332/0 (socket says 172.21.15.129:60332) 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf97fff640 1 -- 172.21.15.129:0/1328775736 learned_addr learned my addr 172.21.15.129:0/1328775736 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf8ffff640 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d67b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf977fe640 1 -- 172.21.15.129:0/1328775736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 msgr2=0x55585d67b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf977fe640 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d67b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf977fe640 1 -- 172.21.15.129:0/1328775736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 msgr2=0x55585d682220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf977fe640 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d682220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf977fe640 1 -- 172.21.15.129:0/1328775736 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55585d40ff70 con 0x55585d6708b0 2024-04-26T21:21:59.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.689+0000 7fdf97fff640 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d682220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdf977fe640 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67aca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf880ce230 tx=0x7fdf88098d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d032fba54585d464 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdf76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdf76ffd640 1 -- 172.21.15.129:0/1328775736 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8809d100 con 0x55585d6708b0 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/1328775736 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf900125a0 con 0x55585d6708b0 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdf76ffd640 1 -- 172.21.15.129:0/1328775736 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fdf880bbbc0 con 0x55585d6708b0 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdf76ffd640 1 -- 172.21.15.129:0/1328775736 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdf8809ca00 con 0x55585d6708b0 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdfa0d6c740 10 client.25427 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/1328775736 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55585d2a7170 con 0x55585d6708b0 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdf76ffd640 1 -- 172.21.15.129:0/1328775736 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf900125a0 con 0x55585d6708b0 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdf76ffd640 1 -- 172.21.15.129:0/1328775736 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fdf88098750 con 0x55585d6708b0 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdfa0d6c740 10 client.25427 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdfa0d6c740 10 client.25427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdfa0d6c740 20 client.25427 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:59.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55585d686760 0x55585d688c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdfa0d6c740 4 client.25427 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:21:59.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.690+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/1328775736 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55585d5e3710 con 0x55585d686760 2024-04-26T21:21:59.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.691+0000 7fdf8ffff640 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55585d686760 0x55585d688c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.691+0000 7fdf8ffff640 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55585d686760 0x55585d688c00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917b99c34d6bddc9 server_cookie=a87ea3e1027a3c10 in_seq=0 out_seq=0 2024-04-26T21:21:59.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.691+0000 7fdf76ffd640 10 client.25427 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:59.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdf76ffd640 1 -- 172.21.15.129:0/1328775736 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55585d5e3710 con 0x55585d686760 2024-04-26T21:21:59.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdf76ffd640 10 client.25427 handle_command_reply: tid=0 2024-04-26T21:21:59.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdfa0d6c740 1 client.25427 shutdown 2024-04-26T21:21:59.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/1328775736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 msgr2=0x55585d67aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67aca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf880ce230 tx=0x7fdf88098d60 comp rx=0 tx=0).stop 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/1328775736 shutdown_connections 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55585d686760 0x55585d688c00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d682220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d67b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdf76ffd640 0 client.25427 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.693+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/1328775736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67aca0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.694+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/1328775736 >> 172.21.15.129:0/1328775736 conn(0x55585d66bf20 msgr2=0x55585d66b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.694+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/1328775736 shutdown_connections 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.694+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/1328775736 wait complete. 2024-04-26T21:21:59.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.694+0000 7fdfa0d6c740 20 client.25427 trim_cache size 0 max 16384 2024-04-26T21:21:59.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.711+0000 7fdfa0d6c740 1 Processor -- start 2024-04-26T21:21:59.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.711+0000 7fdfa0d6c740 1 -- start start 2024-04-26T21:21:59.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d6821e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d67adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7800c3a0 con 0x55585d676410 2024-04-26T21:21:59.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7800ec20 con 0x55585d6708b0 2024-04-26T21:21:59.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7800ca10 con 0x55585d6712a0 2024-04-26T21:21:59.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf97fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d67adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf97fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d67adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39904/0 (socket says 172.21.15.129:39904) 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf97fff640 1 -- 172.21.15.129:0/4030003701 learned_addr learned my addr 172.21.15.129:0/4030003701 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf977fe640 1 --2- 172.21.15.129:0/4030003701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf8ffff640 1 --2- 172.21.15.129:0/4030003701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d6821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf97fff640 1 -- 172.21.15.129:0/4030003701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 msgr2=0x55585d6821e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf97fff640 1 --2- 172.21.15.129:0/4030003701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d6821e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf97fff640 1 -- 172.21.15.129:0/4030003701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 msgr2=0x55585d67a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf97fff640 1 --2- 172.21.15.129:0/4030003701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf97fff640 1 -- 172.21.15.129:0/4030003701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585d66bd50 con 0x55585d676410 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf97fff640 1 --2- 172.21.15.129:0/4030003701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d67adf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdf90005f10 tx=0x7fdf90009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d3690809dc1c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf8d7fa640 1 -- 172.21.15.129:0/4030003701 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90024040 con 0x55585d676410 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf8d7fa640 1 -- 172.21.15.129:0/4030003701 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf900125a0 con 0x55585d676410 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.712+0000 7fdf8d7fa640 1 -- 172.21.15.129:0/4030003701 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90020bd0 con 0x55585d676410 2024-04-26T21:21:59.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/4030003701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 msgr2=0x55585d67adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/4030003701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d67adf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdf90005f10 tx=0x7fdf90009680 comp rx=0 tx=0).stop 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/4030003701 shutdown_connections 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/4030003701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d676410 0x55585d67adf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/4030003701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d6712a0 0x55585d6821e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/4030003701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6708b0 0x55585d67a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/4030003701 >> 172.21.15.129:0/4030003701 conn(0x55585d66bf20 msgr2=0x55585d672700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/4030003701 shutdown_connections 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/4030003701 wait complete. 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.713+0000 7fdfa0d6c740 1 -- start start 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdfa0d6c740 1 Processor -- start 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d6708b0 0x55585d67f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6712a0 0x55585d67fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdfa0d6c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d676410 0x55585d680bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7800d0e0 con 0x55585d6708b0 2024-04-26T21:21:59.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7800e550 con 0x55585d6712a0 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdfa0d6c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7800d7b0 con 0x55585d676410 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdfa0d6c740 10 client.0 _ll_register_callbacks cb 0x7fdf880097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d6708b0 0x55585d67f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d6708b0 0x55585d67f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39906/0 (socket says 172.21.15.129:39906) 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf977fe640 1 -- 172.21.15.129:0/3547737691 learned_addr learned my addr 172.21.15.129:0/3547737691 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf8ffff640 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6712a0 0x55585d67fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf97fff640 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d676410 0x55585d680bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf977fe640 1 -- 172.21.15.129:0/3547737691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d676410 msgr2=0x55585d680bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf977fe640 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d676410 0x55585d680bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf977fe640 1 -- 172.21.15.129:0/3547737691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6712a0 msgr2=0x55585d67fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf977fe640 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6712a0 0x55585d67fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf977fe640 1 -- 172.21.15.129:0/3547737691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55585d6811c0 con 0x55585d6708b0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf977fe640 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d6708b0 0x55585d67f6d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdf880ce7b0 tx=0x7fdf880039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d179de0deb2dab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.714+0000 7fdf76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdf76ffd640 1 -- 172.21.15.129:0/3547737691 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8809c540 con 0x55585d6708b0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdf76ffd640 1 -- 172.21.15.129:0/3547737691 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fdf8809c050 con 0x55585d6708b0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3547737691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf900125a0 con 0x55585d6708b0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdf76ffd640 1 -- 172.21.15.129:0/3547737691 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdf880ac050 con 0x55585d6708b0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdfa0d6c740 10 client.15834 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3547737691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55585d682500 con 0x55585d6708b0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdf76ffd640 1 -- 172.21.15.129:0/3547737691 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf900125a0 con 0x55585d6708b0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdf76ffd640 1 -- 172.21.15.129:0/3547737691 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fdf8809b8c0 con 0x55585d6708b0 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdfa0d6c740 10 client.15834 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:59.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdfa0d6c740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:59.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdfa0d6c740 20 client.15834 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:59.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55585d6ed420 0x55585d6ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdfa0d6c740 4 client.15834 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:21:59.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3547737691 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55585d5dd580 con 0x55585d6ed420 2024-04-26T21:21:59.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.715+0000 7fdf8ffff640 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55585d6ed420 0x55585d6ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.716+0000 7fdf76ffd640 10 client.15834 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:59.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.716+0000 7fdf8ffff640 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55585d6ed420 0x55585d6ef840 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc973b0a66f03f2c server_cookie=a411a661f540f3d0 in_seq=0 out_seq=0 2024-04-26T21:21:59.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:21:59 smithi171 ceph-mon[33478]: pgmap v470: 97 pgs: 97 active+clean; 552 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.5 MiB/s wr, 484 op/s 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdf76ffd640 1 -- 172.21.15.129:0/3547737691 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55585d5dd580 con 0x55585d6ed420 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdf76ffd640 10 client.15834 handle_command_reply: tid=0 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 client.15834 shutdown 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3547737691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d6708b0 msgr2=0x55585d67f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d6708b0 0x55585d67f6d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdf880ce7b0 tx=0x7fdf880039d0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3547737691 shutdown_connections 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55585d6ed420 0x55585d6ef840 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55585d6712a0 0x55585d67fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55585d676410 0x55585d680bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdf76ffd640 0 client.15834 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 --2- 172.21.15.129:0/3547737691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55585d6708b0 0x55585d67f6d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3547737691 >> 172.21.15.129:0/3547737691 conn(0x55585d66bf20 msgr2=0x55585d66f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3547737691 shutdown_connections 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 1 -- 172.21.15.129:0/3547737691 wait complete. 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.776+0000 7fdfa0d6c740 20 client.15834 trim_cache size 0 max 16384 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "5846dc40-4197-47a8-90b0-6ec0e5d15dd0", 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:21:59.779 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:21:59.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 -- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdf840a3cc0 msgr2=0x7fdf840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdf840a3cc0 0x7fdf840a6180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdf90010c40 tx=0x7fdf9000c000 comp rx=0 tx=0).stop 2024-04-26T21:21:59.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 -- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98101870 msgr2=0x7fdf9806a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98101870 0x7fdf9806a630 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800e9c0 tx=0x7fdf8800ee80 comp rx=0 tx=0).stop 2024-04-26T21:21:59.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 -- 172.21.15.129:0/3805111344 shutdown_connections 2024-04-26T21:21:59.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fdf840a3cc0 0x7fdf840a6180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf98103420 0x7fdf9819ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf98102a60 0x7fdf9819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 --2- 172.21.15.129:0/3805111344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf98101870 0x7fdf9806a630 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 -- 172.21.15.129:0/3805111344 >> 172.21.15.129:0/3805111344 conn(0x7fdf980fcde0 msgr2=0x7fdf98105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 -- 172.21.15.129:0/3805111344 shutdown_connections 2024-04-26T21:21:59.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.777+0000 7fdf9df78640 1 -- 172.21.15.129:0/3805111344 wait complete. 2024-04-26T21:21:59.788 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:21:59 smithi129 ceph-mon[29237]: pgmap v470: 97 pgs: 97 active+clean; 552 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.5 MiB/s wr, 484 op/s 2024-04-26T21:21:59.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:21:59.789 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:21:59.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:21:59 smithi150 ceph-mon[32290]: pgmap v470: 97 pgs: 97 active+clean; 552 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.5 MiB/s wr, 484 op/s 2024-04-26T21:21:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.893+0000 7f7286d11640 1 -- 172.21.15.129:0/1414690069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280101870 msgr2=0x7f7280101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.893+0000 7f7286d11640 1 --2- 172.21.15.129:0/1414690069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280101870 0x7f7280101c90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7268009900 tx=0x7f726802a410 comp rx=0 tx=0).stop 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.893+0000 7f7286d11640 1 -- 172.21.15.129:0/1414690069 shutdown_connections 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.893+0000 7f7286d11640 1 --2- 172.21.15.129:0/1414690069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7280102a60 0x7f7280102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.893+0000 7f7286d11640 1 --2- 172.21.15.129:0/1414690069 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7280103420 0x7f7280109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.893+0000 7f7286d11640 1 --2- 172.21.15.129:0/1414690069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280101870 0x7f7280101c90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.893+0000 7f7286d11640 1 -- 172.21.15.129:0/1414690069 >> 172.21.15.129:0/1414690069 conn(0x7f72800fcde0 msgr2=0x7f72800ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.893+0000 7f7286d11640 1 -- 172.21.15.129:0/1414690069 shutdown_connections 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.893+0000 7f7286d11640 1 -- 172.21.15.129:0/1414690069 wait complete. 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f7286d11640 1 Processor -- start 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f7286d11640 1 -- start start 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f7286d11640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7280101870 0x7f7280198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f7286d11640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280102a60 0x7f72801991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f7286d11640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7280103420 0x7f72801a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f7286d11640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f728010e480 con 0x7f7280102a60 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f7286d11640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f728010e300 con 0x7f7280103420 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f7286d11640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f728010e600 con 0x7f7280101870 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f727ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280102a60 0x7f72801991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f727ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280102a60 0x7f72801991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39916/0 (socket says 172.21.15.129:39916) 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f727ffff640 1 -- 172.21.15.129:0/3944509926 learned_addr learned my addr 172.21.15.129:0/3944509926 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f72852ae640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7280103420 0x7f72801a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f7284aad640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7280101870 0x7f7280198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f727ffff640 1 -- 172.21.15.129:0/3944509926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7280101870 msgr2=0x7f7280198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f727ffff640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7280101870 0x7f7280198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f727ffff640 1 -- 172.21.15.129:0/3944509926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7280103420 msgr2=0x7f72801a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f727ffff640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7280103420 0x7f72801a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.894+0000 7f727ffff640 1 -- 172.21.15.129:0/3944509926 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72801a0880 con 0x7f7280102a60 2024-04-26T21:21:59.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.895+0000 7f727ffff640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280102a60 0x7f72801991b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f727000e9f0 tx=0x7f727000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.895+0000 7f727dffb640 1 -- 172.21.15.129:0/3944509926 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727000cd80 con 0x7f7280102a60 2024-04-26T21:21:59.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.895+0000 7f7286d11640 1 -- 172.21.15.129:0/3944509926 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72801a0b70 con 0x7f7280102a60 2024-04-26T21:21:59.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.895+0000 7f727dffb640 1 -- 172.21.15.129:0/3944509926 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72700108e0 con 0x7f7280102a60 2024-04-26T21:21:59.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.895+0000 7f727dffb640 1 -- 172.21.15.129:0/3944509926 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270018980 con 0x7f7280102a60 2024-04-26T21:21:59.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.895+0000 7f7286d11640 1 -- 172.21.15.129:0/3944509926 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72801a1110 con 0x7f7280102a60 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f7289a22740 1 Processor -- start 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f727dffb640 1 -- 172.21.15.129:0/3944509926 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7270010430 con 0x7f7280102a60 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f7289a22740 1 -- start start 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f7289a22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb44710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f7289a22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 0x55bb5cb45940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f7289a22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb4c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f7289a22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bb5caa4d80 con 0x55bb5cb45e80 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f7289a22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bb5ca5e8b0 con 0x55bb5cb442f0 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f7289a22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bb5caa0d40 con 0x55bb5cb454c0 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f727ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb4c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f72852ae640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb44710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f72852ae640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb44710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60378/0 (socket says 172.21.15.129:60378) 2024-04-26T21:21:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f72852ae640 1 -- 172.21.15.129:0/3404382633 learned_addr learned my addr 172.21.15.129:0/3404382633 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f7284aad640 1 --2- 172.21.15.129:0/3404382633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 0x55bb5cb45940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f72852ae640 1 -- 172.21.15.129:0/3404382633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 msgr2=0x55bb5cb45940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f72852ae640 1 --2- 172.21.15.129:0/3404382633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 0x55bb5cb45940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f72852ae640 1 -- 172.21.15.129:0/3404382633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 msgr2=0x55bb5cb4c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.898+0000 7f72852ae640 1 --2- 172.21.15.129:0/3404382633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb4c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.899+0000 7f72852ae640 1 -- 172.21.15.129:0/3404382633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5ca2fa40 con 0x55bb5cb442f0 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.899+0000 7f727dffb640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f726c0a3c40 0x7f726c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.899+0000 7f7284aad640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f726c0a3c40 0x7f726c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.899+0000 7f727dffb640 1 -- 172.21.15.129:0/3944509926 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f72700d0900 con 0x7f7280102a60 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.899+0000 7f72852ae640 1 --2- 172.21.15.129:0/3404382633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb44710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f72780039f0 tx=0x7f7278003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ded2e65654d84d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.899+0000 7f7276ffd640 1 -- 172.21.15.129:0/3404382633 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727801a470 con 0x55bb5cb442f0 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.899+0000 7f7276ffd640 1 -- 172.21.15.129:0/3404382633 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7278015070 con 0x55bb5cb442f0 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.899+0000 7f7276ffd640 1 -- 172.21.15.129:0/3404382633 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72780107e0 con 0x55bb5cb442f0 2024-04-26T21:21:59.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 -- 172.21.15.129:0/3404382633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 msgr2=0x55bb5cb44710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7284aad640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f726c0a3c40 0x7f726c0a6100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f728019a140 tx=0x7f72680314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 --2- 172.21.15.129:0/3404382633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb44710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f72780039f0 tx=0x7f7278003eb0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 -- 172.21.15.129:0/3404382633 shutdown_connections 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 --2- 172.21.15.129:0/3404382633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb4c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 --2- 172.21.15.129:0/3404382633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 0x55bb5cb45940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 --2- 172.21.15.129:0/3404382633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb44710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 -- 172.21.15.129:0/3404382633 >> 172.21.15.129:0/3404382633 conn(0x55bb5cb3b3d0 msgr2=0x55bb5cb3b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 -- 172.21.15.129:0/3404382633 shutdown_connections 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 -- 172.21.15.129:0/3404382633 wait complete. 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 -- start start 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 Processor -- start 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 0x55bb5cb4ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb55d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bb5cb4ce90 con 0x55bb5cb45e80 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bb5ca2bd10 con 0x55bb5cb454c0 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bb5cb4d010 con 0x55bb5cb442f0 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.900+0000 7f7289a22740 10 client.0 _ll_register_callbacks cb 0x55bb5ca58210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f72852ae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb55d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f7284aad640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f727ffff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 0x55bb5cb4ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f72852ae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb55d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39922/0 (socket says 172.21.15.129:39922) 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f7284aad640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34384/0 (socket says 172.21.15.129:34384) 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f72852ae640 1 -- 172.21.15.129:0/2134391099 learned_addr learned my addr 172.21.15.129:0/2134391099 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f72852ae640 1 -- 172.21.15.129:0/2134391099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb442f0 msgr2=0x55bb5cb4e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f72852ae640 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f72852ae640 1 -- 172.21.15.129:0/2134391099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 msgr2=0x55bb5cb4ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f72852ae640 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 0x55bb5cb4ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f72852ae640 1 -- 172.21.15.129:0/2134391099 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5c8807d0 con 0x55bb5cb45e80 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f727ffff640 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 0x55bb5cb4ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f72852ae640 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb55d30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f72780049a0 tx=0x7f7278005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcf39e42888d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f725ffff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f725ffff640 1 -- 172.21.15.129:0/2134391099 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278011650 con 0x55bb5cb45e80 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f725ffff640 1 -- 172.21.15.129:0/2134391099 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f72780117f0 con 0x55bb5cb45e80 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f7289a22740 1 -- 172.21.15.129:0/2134391099 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7278015070 con 0x55bb5cb45e80 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f725ffff640 1 -- 172.21.15.129:0/2134391099 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7278011a30 con 0x55bb5cb45e80 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f7289a22740 10 client.15846 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f7289a22740 1 -- 172.21.15.129:0/2134391099 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bb5c7b63f0 con 0x55bb5cb45e80 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.901+0000 7f725ffff640 1 -- 172.21.15.129:0/2134391099 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7278015070 con 0x55bb5cb45e80 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f725ffff640 1 -- 172.21.15.129:0/2134391099 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f727801a470 con 0x55bb5cb45e80 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f7289a22740 10 client.15846 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f7289a22740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:59.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f7289a22740 20 client.15846 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f7289a22740 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bb5cb59f40 0x55bb5cb5c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f7289a22740 4 client.15846 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f7289a22740 1 -- 172.21.15.129:0/2134391099 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb5cab2990 con 0x55bb5cb59f40 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f7284aad640 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bb5cb59f40 0x55bb5cb5c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f7284aad640 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bb5cb59f40 0x55bb5cb5c330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d4ac29ed95a866 server_cookie=1da5400c218b492d in_seq=0 out_seq=0 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.902+0000 7f725ffff640 10 client.15846 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.903+0000 7f725ffff640 1 -- 172.21.15.129:0/2134391099 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb5cab2990 con 0x55bb5cb59f40 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.903+0000 7f725ffff640 10 client.15846 handle_command_reply: tid=0 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.903+0000 7f7289a22740 1 client.15846 shutdown 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 -- 172.21.15.129:0/2134391099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 msgr2=0x55bb5cb55d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb55d30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f72780049a0 tx=0x7f7278005ac0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 -- 172.21.15.129:0/2134391099 shutdown_connections 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bb5cb59f40 0x55bb5cb5c330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 0x55bb5cb4ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb55d30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f725ffff640 0 client.15846 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:59.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 --2- 172.21.15.129:0/2134391099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 -- 172.21.15.129:0/2134391099 >> 172.21.15.129:0/2134391099 conn(0x55bb5cb3b3d0 msgr2=0x55bb5cb3a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 -- 172.21.15.129:0/2134391099 shutdown_connections 2024-04-26T21:21:59.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 1 -- 172.21.15.129:0/2134391099 wait complete. 2024-04-26T21:21:59.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.904+0000 7f7289a22740 20 client.15846 trim_cache size 0 max 16384 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7289a22740 1 Processor -- start 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7289a22740 1 -- start start 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7289a22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7289a22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 0x55bb5cb55cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7289a22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb45e80 0x55bb5cb4ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7289a22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72801a5c80 con 0x55bb5cb442f0 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7289a22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f72801a8500 con 0x55bb5cb454c0 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7289a22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f72801a62f0 con 0x55bb5cb45e80 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7284aad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7284aad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39928/0 (socket says 172.21.15.129:39928) 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f7284aad640 1 -- 172.21.15.129:0/1585803980 learned_addr learned my addr 172.21.15.129:0/1585803980 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f72852ae640 1 --2- 172.21.15.129:0/1585803980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb45e80 0x55bb5cb4ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.922+0000 7f727ffff640 1 --2- 172.21.15.129:0/1585803980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 0x55bb5cb55cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7284aad640 1 -- 172.21.15.129:0/1585803980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb45e80 msgr2=0x55bb5cb4ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7284aad640 1 --2- 172.21.15.129:0/1585803980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb45e80 0x55bb5cb4ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7284aad640 1 -- 172.21.15.129:0/1585803980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 msgr2=0x55bb5cb55cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7284aad640 1 --2- 172.21.15.129:0/1585803980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 0x55bb5cb55cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7284aad640 1 -- 172.21.15.129:0/1585803980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5c7760f0 con 0x55bb5cb442f0 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7284aad640 1 --2- 172.21.15.129:0/1585803980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f72801028c0 tx=0x7f7268040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d528b68ff333e0c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f72767fc640 1 -- 172.21.15.129:0/1585803980 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268035040 con 0x55bb5cb442f0 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f72767fc640 1 -- 172.21.15.129:0/1585803980 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72680538d0 con 0x55bb5cb442f0 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f72767fc640 1 -- 172.21.15.129:0/1585803980 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268003e40 con 0x55bb5cb442f0 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7289a22740 1 -- 172.21.15.129:0/1585803980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb442f0 msgr2=0x55bb5cb4e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7289a22740 1 --2- 172.21.15.129:0/1585803980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f72801028c0 tx=0x7f7268040000 comp rx=0 tx=0).stop 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7289a22740 1 -- 172.21.15.129:0/1585803980 shutdown_connections 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.923+0000 7f7289a22740 1 --2- 172.21.15.129:0/1585803980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb454c0 0x55bb5cb55cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 --2- 172.21.15.129:0/1585803980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb45e80 0x55bb5cb4ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 --2- 172.21.15.129:0/1585803980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb442f0 0x55bb5cb4e7b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 -- 172.21.15.129:0/1585803980 >> 172.21.15.129:0/1585803980 conn(0x55bb5cb3b3d0 msgr2=0x55bb5cb48f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 -- 172.21.15.129:0/1585803980 shutdown_connections 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 -- 172.21.15.129:0/1585803980 wait complete. 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 -- start start 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 Processor -- start 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 0x55bb5cb53700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72801a69c0 con 0x55bb5cb45e80 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f72801a7e30 con 0x55bb5cb442f0 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f72801a7090 con 0x55bb5cb454c0 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7289a22740 10 client.0 _ll_register_callbacks cb 0x7f72700d6380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:59.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7284aad640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f72852ae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7284aad640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60394/0 (socket says 172.21.15.129:60394) 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f7284aad640 1 -- 172.21.15.129:0/4154108454 learned_addr learned my addr 172.21.15.129:0/4154108454 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.924+0000 7f727ffff640 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 0x55bb5cb53700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f7284aad640 1 -- 172.21.15.129:0/4154108454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 msgr2=0x55bb5cb53700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f7284aad640 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 0x55bb5cb53700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f7284aad640 1 -- 172.21.15.129:0/4154108454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 msgr2=0x55bb5cb546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f7284aad640 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f7284aad640 1 -- 172.21.15.129:0/4154108454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5ca654b0 con 0x55bb5cb442f0 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f72852ae640 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f7284aad640 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb531c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f726802a930 tx=0x7f72680037e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7573d8943b525926 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f725ffff640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f725ffff640 1 -- 172.21.15.129:0/4154108454 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268062060 con 0x55bb5cb442f0 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f725ffff640 1 -- 172.21.15.129:0/4154108454 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7268053420 con 0x55bb5cb442f0 2024-04-26T21:21:59.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f7289a22740 1 -- 172.21.15.129:0/4154108454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72680538d0 con 0x55bb5cb442f0 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f725ffff640 1 -- 172.21.15.129:0/4154108454 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7268035040 con 0x55bb5cb442f0 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f7289a22740 10 client.25451 fetch_fsmap learned FSMap version 19 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f7289a22740 1 -- 172.21.15.129:0/4154108454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bb5cb55e60 con 0x55bb5cb442f0 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.925+0000 7f725ffff640 1 -- 172.21.15.129:0/4154108454 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72680538d0 con 0x55bb5cb442f0 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f725ffff640 1 -- 172.21.15.129:0/4154108454 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f7268050070 con 0x55bb5cb442f0 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f7289a22740 10 client.25451 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f7289a22740 10 client.25451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f7289a22740 20 client.25451 populate_metadata read hostname 'smithi129' 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f7289a22740 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bb5cbc0a30 0x55bb5cbc2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f7289a22740 4 client.25451 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f7289a22740 1 -- 172.21.15.129:0/4154108454 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb5caac800 con 0x55bb5cbc0a30 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f727ffff640 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bb5cbc0a30 0x55bb5cbc2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f727ffff640 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bb5cbc0a30 0x55bb5cbc2e50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf162dbe95d4c769 server_cookie=b4fc1f5e54f133a7 in_seq=0 out_seq=0 2024-04-26T21:21:59.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.926+0000 7f725ffff640 10 client.25451 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f725ffff640 1 -- 172.21.15.129:0/4154108454 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55bb5caac800 con 0x55bb5cbc0a30 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f725ffff640 10 client.25451 handle_command_reply: tid=0 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 client.25451 shutdown 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 -- 172.21.15.129:0/4154108454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 msgr2=0x55bb5cb531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb531c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f726802a930 tx=0x7f72680037e0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 -- 172.21.15.129:0/4154108454 shutdown_connections 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55bb5cbc0a30 0x55bb5cbc2e50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bb5cb45e80 0x55bb5cb546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bb5cb454c0 0x55bb5cb53700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 --2- 172.21.15.129:0/4154108454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb5cb442f0 0x55bb5cb531c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f725ffff640 0 client.25451 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:21:59.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 -- 172.21.15.129:0/4154108454 >> 172.21.15.129:0/4154108454 conn(0x55bb5cb3b3d0 msgr2=0x55bb5cb4e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.929+0000 7f7289a22740 1 -- 172.21.15.129:0/4154108454 shutdown_connections 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7289a22740 1 -- 172.21.15.129:0/4154108454 wait complete. 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7289a22740 20 client.25451 trim_cache size 0 max 16384 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (2 inodes in the stack)", 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stdout: "5846dc40-4197-47a8-90b0-6ec0e5d15dd0": { 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "5846dc40-4197-47a8-90b0-6ec0e5d15dd0", 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:21:59.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:21:59.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:21:59.936 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:21:59.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 -- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f726c0a3c40 msgr2=0x7f726c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f726c0a3c40 0x7f726c0a6100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f728019a140 tx=0x7f72680314a0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 -- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280102a60 msgr2=0x7f72801991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280102a60 0x7f72801991b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f727000e9f0 tx=0x7f727000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 -- 172.21.15.129:0/3944509926 shutdown_connections 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f726c0a3c40 0x7f726c0a6100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7280103420 0x7f72801a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7280102a60 0x7f72801991b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 --2- 172.21.15.129:0/3944509926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7280101870 0x7f7280198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 -- 172.21.15.129:0/3944509926 >> 172.21.15.129:0/3944509926 conn(0x7f72800fcde0 msgr2=0x7f7280105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 -- 172.21.15.129:0/3944509926 shutdown_connections 2024-04-26T21:21:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:21:59.930+0000 7f7286d11640 1 -- 172.21.15.129:0/3944509926 wait complete. 2024-04-26T21:21:59.946 INFO:tasks.cephfs.filesystem:scrub status for tag:5846dc40-4197-47a8-90b0-6ec0e5d15dd0 - {'path': '/', 'tag': '5846dc40-4197-47a8-90b0-6ec0e5d15dd0', 'options': 'recursive,force'} 2024-04-26T21:22:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:00 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:22:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:00 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:22:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:00 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:22:00.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:00 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:22:00.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:00 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:22:00.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:00 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:22:00.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:00 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:22:00.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:00 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:22:00.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:00 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:22:01.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:01 smithi171 ceph-mon[33478]: pgmap v471: 97 pgs: 97 active+clean; 553 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 579 op/s 2024-04-26T21:22:01.768 INFO:tasks.workunit.client.0.smithi129.stdout: 102 1106130 379 746441 331 635104 1211 2024-04-26T21:22:01.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:01 smithi150 ceph-mon[32290]: pgmap v471: 97 pgs: 97 active+clean; 553 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 579 op/s 2024-04-26T21:22:01.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:01 smithi129 ceph-mon[29237]: pgmap v471: 97 pgs: 97 active+clean; 553 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 579 op/s 2024-04-26T21:22:03.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:03 smithi171 ceph-mon[33478]: pgmap v472: 97 pgs: 97 active+clean; 551 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 534 op/s 2024-04-26T21:22:03.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:03 smithi150 ceph-mon[32290]: pgmap v472: 97 pgs: 97 active+clean; 551 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 534 op/s 2024-04-26T21:22:03.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:03 smithi129 ceph-mon[29237]: pgmap v472: 97 pgs: 97 active+clean; 551 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 534 op/s 2024-04-26T21:22:05.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:05 smithi171 ceph-mon[33478]: pgmap v473: 97 pgs: 97 active+clean; 553 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-04-26T21:22:05.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:05 smithi150 ceph-mon[32290]: pgmap v473: 97 pgs: 97 active+clean; 553 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-04-26T21:22:05.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:05 smithi129 ceph-mon[29237]: pgmap v473: 97 pgs: 97 active+clean; 553 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-04-26T21:22:07.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:07 smithi171 ceph-mon[33478]: pgmap v474: 97 pgs: 97 active+clean; 555 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 386 op/s 2024-04-26T21:22:07.773 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:07 smithi129 ceph-mon[29237]: pgmap v474: 97 pgs: 97 active+clean; 555 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 386 op/s 2024-04-26T21:22:07.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:07 smithi150 ceph-mon[32290]: pgmap v474: 97 pgs: 97 active+clean; 555 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 386 op/s 2024-04-26T21:22:09.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:09 smithi171 ceph-mon[33478]: pgmap v475: 97 pgs: 97 active+clean; 558 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 401 op/s 2024-04-26T21:22:09.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:09 smithi150 ceph-mon[32290]: pgmap v475: 97 pgs: 97 active+clean; 558 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 401 op/s 2024-04-26T21:22:09.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:09 smithi129 ceph-mon[29237]: pgmap v475: 97 pgs: 97 active+clean; 558 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 401 op/s 2024-04-26T21:22:11.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:11 smithi171 ceph-mon[33478]: pgmap v476: 97 pgs: 97 active+clean; 560 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 412 op/s 2024-04-26T21:22:11.768 INFO:tasks.workunit.client.0.smithi129.stdout: 108 2093102 523 1457351 385 1208619 1355 2024-04-26T21:22:11.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:11 smithi150 ceph-mon[32290]: pgmap v476: 97 pgs: 97 active+clean; 560 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 412 op/s 2024-04-26T21:22:11.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:11 smithi129 ceph-mon[29237]: pgmap v476: 97 pgs: 97 active+clean; 560 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 412 op/s 2024-04-26T21:22:13.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:13 smithi171 ceph-mon[33478]: pgmap v477: 97 pgs: 97 active+clean; 560 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 330 op/s 2024-04-26T21:22:13.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:13 smithi150 ceph-mon[32290]: pgmap v477: 97 pgs: 97 active+clean; 560 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 330 op/s 2024-04-26T21:22:13.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:13 smithi129 ceph-mon[29237]: pgmap v477: 97 pgs: 97 active+clean; 560 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 330 op/s 2024-04-26T21:22:15.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:15 smithi171 ceph-mon[33478]: pgmap v478: 97 pgs: 97 active+clean; 563 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 282 op/s 2024-04-26T21:22:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:15 smithi150 ceph-mon[32290]: pgmap v478: 97 pgs: 97 active+clean; 563 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 282 op/s 2024-04-26T21:22:15.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:15 smithi129 ceph-mon[29237]: pgmap v478: 97 pgs: 97 active+clean; 563 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 282 op/s 2024-04-26T21:22:18.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:18 smithi129 ceph-mon[29237]: pgmap v479: 97 pgs: 97 active+clean; 562 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 303 op/s 2024-04-26T21:22:18.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:18 smithi171 ceph-mon[33478]: pgmap v479: 97 pgs: 97 active+clean; 562 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 303 op/s 2024-04-26T21:22:18.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:18 smithi150 ceph-mon[32290]: pgmap v479: 97 pgs: 97 active+clean; 562 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 303 op/s 2024-04-26T21:22:20.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:20 smithi129 ceph-mon[29237]: pgmap v480: 97 pgs: 97 active+clean; 581 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 453 op/s 2024-04-26T21:22:20.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:20 smithi171 ceph-mon[33478]: pgmap v480: 97 pgs: 97 active+clean; 581 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 453 op/s 2024-04-26T21:22:20.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:20 smithi150 ceph-mon[32290]: pgmap v480: 97 pgs: 97 active+clean; 581 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 453 op/s 2024-04-26T21:22:21.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:21 smithi129 ceph-mon[29237]: pgmap v481: 97 pgs: 97 active+clean; 607 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 637 op/s 2024-04-26T21:22:21.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:21 smithi171 ceph-mon[33478]: pgmap v481: 97 pgs: 97 active+clean; 607 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 637 op/s 2024-04-26T21:22:21.768 INFO:tasks.workunit.client.0.smithi129.stdout: 112 1259126 257 869337 172 724033 735 2024-04-26T21:22:21.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:21 smithi150 ceph-mon[32290]: pgmap v481: 97 pgs: 97 active+clean; 607 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 637 op/s 2024-04-26T21:22:24.722 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:24 smithi129 ceph-mon[29237]: pgmap v482: 97 pgs: 97 active+clean; 610 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.1 MiB/s wr, 722 op/s 2024-04-26T21:22:24.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:24 smithi171 ceph-mon[33478]: pgmap v482: 97 pgs: 97 active+clean; 610 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.1 MiB/s wr, 722 op/s 2024-04-26T21:22:24.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:24 smithi150 ceph-mon[32290]: pgmap v482: 97 pgs: 97 active+clean; 610 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.1 MiB/s wr, 722 op/s 2024-04-26T21:22:25.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:25 smithi129 ceph-mon[29237]: pgmap v483: 97 pgs: 97 active+clean; 615 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.6 MiB/s wr, 778 op/s 2024-04-26T21:22:25.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:25 smithi171 ceph-mon[33478]: pgmap v483: 97 pgs: 97 active+clean; 615 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.6 MiB/s wr, 778 op/s 2024-04-26T21:22:25.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:25 smithi150 ceph-mon[32290]: pgmap v483: 97 pgs: 97 active+clean; 615 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.6 MiB/s wr, 778 op/s 2024-04-26T21:22:27.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:27 smithi171 ceph-mon[33478]: pgmap v484: 97 pgs: 97 active+clean; 622 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 793 op/s 2024-04-26T21:22:27.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:27 smithi150 ceph-mon[32290]: pgmap v484: 97 pgs: 97 active+clean; 622 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 793 op/s 2024-04-26T21:22:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:27 smithi129 ceph-mon[29237]: pgmap v484: 97 pgs: 97 active+clean; 622 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 793 op/s 2024-04-26T21:22:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:28 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:28.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:28 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:29.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:28 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:29.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:22:29.947 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:22:29.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:29 smithi171 ceph-mon[33478]: pgmap v485: 97 pgs: 97 active+clean; 626 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 736 op/s 2024-04-26T21:22:30.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:29 smithi150 ceph-mon[32290]: pgmap v485: 97 pgs: 97 active+clean; 626 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 736 op/s 2024-04-26T21:22:30.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:29 smithi129 ceph-mon[29237]: pgmap v485: 97 pgs: 97 active+clean; 626 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 736 op/s 2024-04-26T21:22:30.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.855+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/4198249817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2f810d020 msgr2=0x7ff2f810f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:30.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.855+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/4198249817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2f810d020 0x7ff2f810f490 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009950 tx=0x7ff2e802a480 comp rx=0 tx=0).stop 2024-04-26T21:22:30.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.857+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/4198249817 shutdown_connections 2024-04-26T21:22:30.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.857+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/4198249817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2f810fb20 0x7ff2f8111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.857+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/4198249817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2f810d020 0x7ff2f810f490 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.857+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/4198249817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f8074770 0x7ff2f810ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.857+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/4198249817 >> 172.21.15.129:0/4198249817 conn(0x7ff2f806f780 msgr2=0x7ff2f8071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:30.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.872+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/4198249817 shutdown_connections 2024-04-26T21:22:30.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.872+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/4198249817 wait complete. 2024-04-26T21:22:30.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2fcfbf640 1 Processor -- start 2024-04-26T21:22:30.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2fcfbf640 1 -- start start 2024-04-26T21:22:30.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2fcfbf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f810d020 0x7ff2f81a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2fcfbf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2f810fb20 0x7ff2f81a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2fcfbf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2f81a68b0 0x7ff2f81a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2fcfbf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8115b40 con 0x7ff2f810fb20 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2fcfbf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81159c0 con 0x7ff2f810d020 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2fcfbf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8115cc0 con 0x7ff2f81a68b0 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f810d020 0x7ff2f81a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f810d020 0x7ff2f81a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:55642/0 (socket says 172.21.15.129:55642) 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.873+0000 7ff2f659c640 1 -- 172.21.15.129:0/1255594415 learned_addr learned my addr 172.21.15.129:0/1255594415 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.874+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2f810fb20 0x7ff2f81a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.874+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2f81a68b0 0x7ff2f81a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.874+0000 7ff2f6d9d640 1 -- 172.21.15.129:0/1255594415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f810d020 msgr2=0x7ff2f81a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:30.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.874+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f810d020 0x7ff2f81a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.874+0000 7ff2f6d9d640 1 -- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2f810fb20 msgr2=0x7ff2f81a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:30.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.874+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2f810fb20 0x7ff2f81a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.874+0000 7ff2f6d9d640 1 -- 172.21.15.129:0/1255594415 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f81a9310 con 0x7ff2f81a68b0 2024-04-26T21:22:30.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.874+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2f810fb20 0x7ff2f81a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:22:30.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.874+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2f81a68b0 0x7ff2f81a8ca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000be00 tx=0x7ff2f000bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:30.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.892+0000 7ff2e77fe640 1 -- 172.21.15.129:0/1255594415 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0019070 con 0x7ff2f81a68b0 2024-04-26T21:22:30.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.892+0000 7ff2e77fe640 1 -- 172.21.15.129:0/1255594415 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f00148f0 con 0x7ff2f81a68b0 2024-04-26T21:22:30.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.892+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/1255594415 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f81a9600 con 0x7ff2f81a68b0 2024-04-26T21:22:30.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.892+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/1255594415 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f8110240 con 0x7ff2f81a68b0 2024-04-26T21:22:30.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.894+0000 7ff2e77fe640 1 -- 172.21.15.129:0/1255594415 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f00128f0 con 0x7ff2f81a68b0 2024-04-26T21:22:30.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.896+0000 7ff2ffdbb740 1 Processor -- start 2024-04-26T21:22:30.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.898+0000 7ff2ffdbb740 1 -- start start 2024-04-26T21:22:30.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ecec0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 0x5648ecec1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec2450 0x5648ecec8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2f5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ecec0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2f5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ecec0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53040/0 (socket says 172.21.15.129:53040) 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/4179829665 learned_addr learned my addr 172.21.15.129:0/4179829665 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2f659c640 1 --2- 172.21.15.129:0/4179829665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 0x5648ecec1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/4179829665 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648ece1cd80 con 0x5648ecec08c0 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/4179829665 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5648ecdd68b0 con 0x5648ecec2450 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/4179829665 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5648ece18d40 con 0x5648ecec1a90 2024-04-26T21:22:30.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.899+0000 7ff2e77fe640 1 -- 172.21.15.129:0/1255594415 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff2f001c070 con 0x7ff2f81a68b0 2024-04-26T21:22:30.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.901+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/4179829665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec2450 0x5648ecec8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.902+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/4179829665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 msgr2=0x5648ecec1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:30.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.902+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/4179829665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 0x5648ecec1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.902+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/4179829665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec2450 msgr2=0x5648ecec8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:30.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.902+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/4179829665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec2450 0x5648ecec8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.902+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/4179829665 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ecda7a40 con 0x5648ecec08c0 2024-04-26T21:22:30.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.902+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/4179829665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ecec0ce0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005dc0 tx=0x7ff2e8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4372717f0f2a177e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:30.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.902+0000 7ff2e77fe640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2e00a3e40 0x7ff2e00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.903+0000 7ff2f659c640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2e00a3e40 0x7ff2e00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.903+0000 7ff2e77fe640 1 -- 172.21.15.129:0/1255594415 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff2f00cfb20 con 0x7ff2f81a68b0 2024-04-26T21:22:30.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.903+0000 7ff2f659c640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2e00a3e40 0x7ff2e00a6300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009bf0 tx=0x7ff2ec009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:30.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.908+0000 7ff2e4ff9640 1 -- 172.21.15.129:0/4179829665 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8035070 con 0x5648ecec08c0 2024-04-26T21:22:30.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.908+0000 7ff2e4ff9640 1 -- 172.21.15.129:0/4179829665 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e80042e0 con 0x5648ecec08c0 2024-04-26T21:22:30.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.908+0000 7ff2e4ff9640 1 -- 172.21.15.129:0/4179829665 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e803c740 con 0x5648ecec08c0 2024-04-26T21:22:30.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.909+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/4179829665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 msgr2=0x5648ecec0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:30.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.909+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/4179829665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ecec0ce0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005dc0 tx=0x7ff2e8032f40 comp rx=0 tx=0).stop 2024-04-26T21:22:30.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.909+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/4179829665 shutdown_connections 2024-04-26T21:22:30.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.910+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/4179829665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec2450 0x5648ecec8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.910+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/4179829665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 0x5648ecec1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.910+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/4179829665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ecec0ce0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.910+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/4179829665 >> 172.21.15.129:0/4179829665 conn(0x5648eceb35a0 msgr2=0x5648eceb39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:30.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.910+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/4179829665 shutdown_connections 2024-04-26T21:22:30.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.910+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/4179829665 wait complete. 2024-04-26T21:22:30.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.910+0000 7ff2ffdbb740 1 -- start start 2024-04-26T21:22:30.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.910+0000 7ff2ffdbb740 1 Processor -- start 2024-04-26T21:22:30.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec1a90 0x5648ececb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648eced2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2ffdbb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648ecec9400 con 0x5648ecec1a90 2024-04-26T21:22:30.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2ffdbb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5648ecda3d10 con 0x5648ecec08c0 2024-04-26T21:22:30.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2ffdbb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5648ecec9580 con 0x5648ecec2450 2024-04-26T21:22:30.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2ffdbb740 10 client.0 _ll_register_callbacks cb 0x5648ecdd0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:55648/0 (socket says 172.21.15.129:55648) 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2f659c640 1 -- 172.21.15.129:0/1498600477 learned_addr learned my addr 172.21.15.129:0/1498600477 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2f659c640 1 -- 172.21.15.129:0/1498600477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 msgr2=0x5648eced2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648eced2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2f659c640 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648eced2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.911+0000 7ff2f659c640 1 -- 172.21.15.129:0/1498600477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec1a90 msgr2=0x5648ececb2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.912+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec1a90 0x5648ececb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.912+0000 7ff2f659c640 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec1a90 0x5648ececb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.912+0000 7ff2f659c640 1 -- 172.21.15.129:0/1498600477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ecbf87d0 con 0x5648ecec08c0 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.912+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648eced2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.912+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec1a90 0x5648ececb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:22:30.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.912+0000 7ff2f659c640 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececadb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec014970 tx=0x7ff2ec0046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4c582b0b31d6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:30.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.913+0000 7ff2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:22:30.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.913+0000 7ff2ddffb640 1 -- 172.21.15.129:0/1498600477 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec025070 con 0x5648ecec08c0 2024-04-26T21:22:30.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.913+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1498600477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e80042e0 con 0x5648ecec08c0 2024-04-26T21:22:30.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.913+0000 7ff2ddffb640 1 -- 172.21.15.129:0/1498600477 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff2ec013670 con 0x5648ecec08c0 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.913+0000 7ff2ddffb640 1 -- 172.21.15.129:0/1498600477 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff2ec031050 con 0x5648ecec08c0 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.913+0000 7ff2ffdbb740 10 client.25460 fetch_fsmap learned FSMap version 19 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.913+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1498600477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5648ecb2e3f0 con 0x5648ecec08c0 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.913+0000 7ff2ddffb640 1 -- 172.21.15.129:0/1498600477 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e80042e0 con 0x5648ecec08c0 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2ddffb640 1 -- 172.21.15.129:0/1498600477 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ff2ec020070 con 0x5648ecec08c0 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2ffdbb740 10 client.25460 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2ffdbb740 10 client.25460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2ffdbb740 20 client.25460 populate_metadata read hostname 'smithi129' 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5648eced6ce0 0x5648eced9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2ffdbb740 4 client.25460 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1498600477 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648ece2a990 con 0x5648eced6ce0 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5648eced6ce0 0x5648eced9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2ddffb640 10 client.25460 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:22:30.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.914+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5648eced6ce0 0x5648eced9180 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6728025f4afd908 server_cookie=6ebe162ac84c2345 in_seq=0 out_seq=0 2024-04-26T21:22:30.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ddffb640 1 -- 172.21.15.129:0/1498600477 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648ece2a990 con 0x5648eced6ce0 2024-04-26T21:22:30.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ddffb640 10 client.25460 handle_command_reply: tid=0 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 client.25460 shutdown 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1498600477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 msgr2=0x5648ececadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececadb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec014970 tx=0x7ff2ec0046e0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1498600477 shutdown_connections 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5648eced6ce0 0x5648eced9180 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648eced2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec1a90 0x5648ececb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1498600477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececadb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1498600477 >> 172.21.15.129:0/1498600477 conn(0x5648eceb35a0 msgr2=0x5648eceb26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ddffb640 0 client.25460 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.918+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1498600477 shutdown_connections 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.919+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1498600477 wait complete. 2024-04-26T21:22:30.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.919+0000 7ff2ffdbb740 20 client.25460 trim_cache size 0 max 16384 2024-04-26T21:22:30.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.954+0000 7ff2ffdbb740 1 Processor -- start 2024-04-26T21:22:30.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.955+0000 7ff2ffdbb740 1 -- start start 2024-04-26T21:22:30.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.955+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.955+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 0x5648eced22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.955+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.956+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:55662/0 (socket says 172.21.15.129:55662) 2024-04-26T21:22:30.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.956+0000 7ff2f659c640 1 -- 172.21.15.129:0/1474765011 learned_addr learned my addr 172.21.15.129:0/1474765011 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:22:30.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.956+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec2450 0x5648ececaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.956+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 0x5648eced22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.956+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec2450 0x5648ececaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.956+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1474765011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d800c510 con 0x5648ecec2450 2024-04-26T21:22:30.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.956+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1474765011 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d800ed90 con 0x5648ecec08c0 2024-04-26T21:22:30.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.957+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1474765011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d800cb80 con 0x5648ecec1a90 2024-04-26T21:22:30.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.957+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/1474765011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 msgr2=0x5648ececa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:30.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.957+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.957+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/1474765011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec2450 msgr2=0x5648ececaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:30.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.957+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec2450 0x5648ececaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.957+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/1474765011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ecaee0f0 con 0x5648ecec1a90 2024-04-26T21:22:30.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.957+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec2450 0x5648ececaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:22:30.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.958+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 0x5648eced22f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8031820 tx=0x7ff2e8005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c79f57503dd27dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:30.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.958+0000 7ff2dffff640 1 -- 172.21.15.129:0/1474765011 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8035040 con 0x5648ecec1a90 2024-04-26T21:22:30.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.958+0000 7ff2dffff640 1 -- 172.21.15.129:0/1474765011 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e8057070 con 0x5648ecec1a90 2024-04-26T21:22:30.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.958+0000 7ff2dffff640 1 -- 172.21.15.129:0/1474765011 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004920 con 0x5648ecec1a90 2024-04-26T21:22:30.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1474765011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 msgr2=0x5648eced22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:30.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 0x5648eced22f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8031820 tx=0x7ff2e8005b50 comp rx=0 tx=0).stop 2024-04-26T21:22:30.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1474765011 shutdown_connections 2024-04-26T21:22:30.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec2450 0x5648ececaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec1a90 0x5648eced22f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/1474765011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec08c0 0x5648ececa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1474765011 >> 172.21.15.129:0/1474765011 conn(0x5648eceb35a0 msgr2=0x5648ecec55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:30.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1474765011 shutdown_connections 2024-04-26T21:22:30.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/1474765011 wait complete. 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 -- start start 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.959+0000 7ff2ffdbb740 1 Processor -- start 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ececf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec1a90 0x5648ececf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2ffdbb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648ecf3b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2ffdbb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d800d250 con 0x5648ecec08c0 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2ffdbb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d800e6c0 con 0x5648ecec1a90 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2ffdbb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d800d920 con 0x5648ecec2450 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2ffdbb740 10 client.0 _ll_register_callbacks cb 0x7ff2ec009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f5d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec1a90 0x5648ececf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f5d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec1a90 0x5648ececf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:55670/0 (socket says 172.21.15.129:55670) 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/20142255 learned_addr learned my addr 172.21.15.129:0/20142255 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f659c640 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ececf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648ecf3b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/20142255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 msgr2=0x5648ecf3b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648ecf3b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/20142255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 msgr2=0x5648ececf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ececf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f5d9b640 1 -- 172.21.15.129:0/20142255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ecddd4b0 con 0x5648ecec1a90 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f6d9d640 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648ecf3b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.960+0000 7ff2f659c640 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ececf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.961+0000 7ff2f5d9b640 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec1a90 0x5648ececf620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005d90 tx=0x7ff2e8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fd7e1174c96cbab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.961+0000 7ff2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.961+0000 7ff2ddffb640 1 -- 172.21.15.129:0/20142255 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8060060 con 0x5648ecec1a90 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.961+0000 7ff2ddffb640 1 -- 172.21.15.129:0/20142255 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff2e8005250 con 0x5648ecec1a90 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.961+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/20142255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e8057070 con 0x5648ecec1a90 2024-04-26T21:22:30.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.961+0000 7ff2ddffb640 1 -- 172.21.15.129:0/20142255 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff2e8035040 con 0x5648ecec1a90 2024-04-26T21:22:30.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.961+0000 7ff2ffdbb740 10 client.25466 fetch_fsmap learned FSMap version 19 2024-04-26T21:22:30.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.961+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/20142255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5648ecf3c790 con 0x5648ecec1a90 2024-04-26T21:22:30.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.962+0000 7ff2ddffb640 1 -- 172.21.15.129:0/20142255 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e8057070 con 0x5648ecec1a90 2024-04-26T21:22:30.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.962+0000 7ff2ddffb640 1 -- 172.21.15.129:0/20142255 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ff2e8064020 con 0x5648ecec1a90 2024-04-26T21:22:30.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.963+0000 7ff2ffdbb740 10 client.25466 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:22:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.963+0000 7ff2ffdbb740 10 client.25466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:22:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.963+0000 7ff2ffdbb740 20 client.25466 populate_metadata read hostname 'smithi129' 2024-04-26T21:22:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.963+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5648ecf3ccb0 0x5648ecf3f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.963+0000 7ff2ffdbb740 4 client.25466 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:22:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.963+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/20142255 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648ece24800 con 0x5648ecf3ccb0 2024-04-26T21:22:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.963+0000 7ff2f659c640 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5648ecf3ccb0 0x5648ecf3f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.963+0000 7ff2ddffb640 10 client.25466 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:22:30.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.963+0000 7ff2f659c640 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5648ecf3ccb0 0x5648ecf3f0a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541b9891ef0e2d13 server_cookie=143b58823772b3a1 in_seq=0 out_seq=0 2024-04-26T21:22:30.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.982+0000 7ff2ddffb640 1 -- 172.21.15.129:0/20142255 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x5648ece24800 con 0x5648ecf3ccb0 2024-04-26T21:22:30.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.982+0000 7ff2ddffb640 10 client.25466 handle_command_reply: tid=0 2024-04-26T21:22:30.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.982+0000 7ff2ffdbb740 1 client.25466 shutdown 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/20142255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec1a90 msgr2=0x5648ececf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec1a90 0x5648ececf620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005d90 tx=0x7ff2e8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/20142255 shutdown_connections 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5648ecf3ccb0 0x5648ecf3f0a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648ecec1a90 0x5648ececf620 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ddffb640 0 client.25466 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648ecec2450 0x5648ecf3b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 --2- 172.21.15.129:0/20142255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648ecec08c0 0x5648ececf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/20142255 >> 172.21.15.129:0/20142255 conn(0x5648eceb35a0 msgr2=0x5648ececa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/20142255 shutdown_connections 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 1 -- 172.21.15.129:0/20142255 wait complete. 2024-04-26T21:22:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2ffdbb740 20 client.25466 trim_cache size 0 max 16384 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (14086 inodes in the stack)", 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "5846dc40-4197-47a8-90b0-6ec0e5d15dd0": { 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "5846dc40-4197-47a8-90b0-6ec0e5d15dd0", 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:22:30.986 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:22:30.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2e00a3e40 msgr2=0x7ff2e00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:30.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2e00a3e40 0x7ff2e00a6300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009bf0 tx=0x7ff2ec009190 comp rx=0 tx=0).stop 2024-04-26T21:22:30.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/1255594415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2f81a68b0 msgr2=0x7ff2f81a8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:30.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2f81a68b0 0x7ff2f81a8ca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000be00 tx=0x7ff2f000bef0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/1255594415 shutdown_connections 2024-04-26T21:22:30.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2e00a3e40 0x7ff2e00a6300 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2f81a68b0 0x7ff2f81a8ca0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2f810fb20 0x7ff2f81a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 --2- 172.21.15.129:0/1255594415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f810d020 0x7ff2f81a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:30.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.983+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/1255594415 >> 172.21.15.129:0/1255594415 conn(0x7ff2f806f780 msgr2=0x7ff2f810bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:30.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.984+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/1255594415 shutdown_connections 2024-04-26T21:22:30.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:22:30.984+0000 7ff2fcfbf640 1 -- 172.21.15.129:0/1255594415 wait complete. 2024-04-26T21:22:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:22:31.003 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:31.003 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:31.003 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:22:31.004 INFO:tasks.cephfs.filesystem:scrub status for tag:5846dc40-4197-47a8-90b0-6ec0e5d15dd0 - {'path': '/', 'tag': '5846dc40-4197-47a8-90b0-6ec0e5d15dd0', 'options': 'recursive,force'} 2024-04-26T21:22:31.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:31.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:31.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:22:31.769 INFO:tasks.workunit.client.0.smithi129.stdout: 118 1668027 666 1119420 270 947760 1511 2024-04-26T21:22:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:31 smithi171 ceph-mon[33478]: pgmap v486: 97 pgs: 97 active+clean; 627 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 562 op/s 2024-04-26T21:22:32.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:31 smithi150 ceph-mon[32290]: pgmap v486: 97 pgs: 97 active+clean; 627 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 562 op/s 2024-04-26T21:22:32.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:31 smithi129 ceph-mon[29237]: pgmap v486: 97 pgs: 97 active+clean; 627 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 562 op/s 2024-04-26T21:22:33.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:33 smithi171 ceph-mon[33478]: pgmap v487: 97 pgs: 97 active+clean; 619 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 349 op/s 2024-04-26T21:22:33.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:33 smithi150 ceph-mon[32290]: pgmap v487: 97 pgs: 97 active+clean; 619 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 349 op/s 2024-04-26T21:22:33.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:33 smithi129 ceph-mon[29237]: pgmap v487: 97 pgs: 97 active+clean; 619 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 349 op/s 2024-04-26T21:22:35.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:35 smithi171 ceph-mon[33478]: pgmap v488: 97 pgs: 97 active+clean; 614 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 233 op/s 2024-04-26T21:22:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:35 smithi150 ceph-mon[32290]: pgmap v488: 97 pgs: 97 active+clean; 614 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 233 op/s 2024-04-26T21:22:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:35 smithi129 ceph-mon[29237]: pgmap v488: 97 pgs: 97 active+clean; 614 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 233 op/s 2024-04-26T21:22:37.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:37 smithi171 ceph-mon[33478]: pgmap v489: 97 pgs: 97 active+clean; 604 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-04-26T21:22:37.773 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:37 smithi129 ceph-mon[29237]: pgmap v489: 97 pgs: 97 active+clean; 604 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-04-26T21:22:37.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:37 smithi150 ceph-mon[32290]: pgmap v489: 97 pgs: 97 active+clean; 604 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-04-26T21:22:39.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:39 smithi150 ceph-mon[32290]: pgmap v490: 97 pgs: 97 active+clean; 611 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 240 op/s 2024-04-26T21:22:39.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:39 smithi129 ceph-mon[29237]: pgmap v490: 97 pgs: 97 active+clean; 611 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 240 op/s 2024-04-26T21:22:39.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:39 smithi171 ceph-mon[33478]: pgmap v490: 97 pgs: 97 active+clean; 611 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 240 op/s 2024-04-26T21:22:41.769 INFO:tasks.workunit.client.0.smithi129.stdout: 123 2081824 460 1396792 378 1178137 1441 2024-04-26T21:22:41.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:41 smithi150 ceph-mon[32290]: pgmap v491: 97 pgs: 97 active+clean; 614 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-04-26T21:22:41.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:41 smithi129 ceph-mon[29237]: pgmap v491: 97 pgs: 97 active+clean; 614 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-04-26T21:22:41.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:41 smithi171 ceph-mon[33478]: pgmap v491: 97 pgs: 97 active+clean; 614 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-04-26T21:22:44.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:44 smithi129 ceph-mon[29237]: pgmap v492: 97 pgs: 97 active+clean; 637 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 482 op/s 2024-04-26T21:22:44.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:44 smithi171 ceph-mon[33478]: pgmap v492: 97 pgs: 97 active+clean; 637 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 482 op/s 2024-04-26T21:22:44.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:44 smithi150 ceph-mon[32290]: pgmap v492: 97 pgs: 97 active+clean; 637 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 482 op/s 2024-04-26T21:22:45.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:45 smithi150 ceph-mon[32290]: pgmap v493: 97 pgs: 97 active+clean; 654 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 607 op/s 2024-04-26T21:22:45.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:45 smithi129 ceph-mon[29237]: pgmap v493: 97 pgs: 97 active+clean; 654 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 607 op/s 2024-04-26T21:22:45.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:45 smithi171 ceph-mon[33478]: pgmap v493: 97 pgs: 97 active+clean; 654 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 607 op/s 2024-04-26T21:22:47.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:47 smithi150 ceph-mon[32290]: pgmap v494: 97 pgs: 97 active+clean; 675 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 760 op/s 2024-04-26T21:22:47.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:47 smithi129 ceph-mon[29237]: pgmap v494: 97 pgs: 97 active+clean; 675 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 760 op/s 2024-04-26T21:22:47.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:47 smithi171 ceph-mon[33478]: pgmap v494: 97 pgs: 97 active+clean; 675 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 760 op/s 2024-04-26T21:22:49.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:49 smithi150 ceph-mon[32290]: pgmap v495: 97 pgs: 97 active+clean; 678 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 849 op/s 2024-04-26T21:22:49.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:49 smithi129 ceph-mon[29237]: pgmap v495: 97 pgs: 97 active+clean; 678 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 849 op/s 2024-04-26T21:22:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:49 smithi171 ceph-mon[33478]: pgmap v495: 97 pgs: 97 active+clean; 678 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 849 op/s 2024-04-26T21:22:51.786 INFO:tasks.workunit.client.0.smithi129.stdout: 127 1147416 300 770530 247 628253 677 2024-04-26T21:22:51.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:51 smithi150 ceph-mon[32290]: pgmap v496: 97 pgs: 97 active+clean; 674 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 919 op/s 2024-04-26T21:22:51.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:51 smithi129 ceph-mon[29237]: pgmap v496: 97 pgs: 97 active+clean; 674 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 919 op/s 2024-04-26T21:22:51.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:51 smithi171 ceph-mon[33478]: pgmap v496: 97 pgs: 97 active+clean; 674 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 919 op/s 2024-04-26T21:22:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:53 smithi150 ceph-mon[32290]: pgmap v497: 97 pgs: 97 active+clean; 657 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 952 op/s 2024-04-26T21:22:53.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:53 smithi129 ceph-mon[29237]: pgmap v497: 97 pgs: 97 active+clean; 657 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 952 op/s 2024-04-26T21:22:53.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:53 smithi171 ceph-mon[33478]: pgmap v497: 97 pgs: 97 active+clean; 657 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 952 op/s 2024-04-26T21:22:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:55 smithi150 ceph-mon[32290]: pgmap v498: 97 pgs: 97 active+clean; 656 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 773 op/s 2024-04-26T21:22:55.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:55 smithi129 ceph-mon[29237]: pgmap v498: 97 pgs: 97 active+clean; 656 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 773 op/s 2024-04-26T21:22:55.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:55 smithi171 ceph-mon[33478]: pgmap v498: 97 pgs: 97 active+clean; 656 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 773 op/s 2024-04-26T21:22:57.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:57 smithi150 ceph-mon[32290]: pgmap v499: 97 pgs: 97 active+clean; 662 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 664 op/s 2024-04-26T21:22:57.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:57 smithi129 ceph-mon[29237]: pgmap v499: 97 pgs: 97 active+clean; 662 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 664 op/s 2024-04-26T21:22:57.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:57 smithi171 ceph-mon[33478]: pgmap v499: 97 pgs: 97 active+clean; 662 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 664 op/s 2024-04-26T21:22:59.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:22:59 smithi150 ceph-mon[32290]: pgmap v500: 97 pgs: 97 active+clean; 661 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 514 op/s 2024-04-26T21:22:59.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:22:59 smithi129 ceph-mon[29237]: pgmap v500: 97 pgs: 97 active+clean; 661 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 514 op/s 2024-04-26T21:22:59.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:22:59 smithi171 ceph-mon[33478]: pgmap v500: 97 pgs: 97 active+clean; 661 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 514 op/s 2024-04-26T21:23:01.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:23:01.006 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:23:01.786 INFO:tasks.workunit.client.0.smithi129.stdout: 136 2316307 721 1548030 345 1250372 1380 2024-04-26T21:23:01.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:01 smithi150 ceph-mon[32290]: pgmap v501: 97 pgs: 97 active+clean; 668 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 388 op/s 2024-04-26T21:23:01.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:01 smithi129 ceph-mon[29237]: pgmap v501: 97 pgs: 97 active+clean; 668 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 388 op/s 2024-04-26T21:23:01.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:01 smithi171 ceph-mon[33478]: pgmap v501: 97 pgs: 97 active+clean; 668 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 388 op/s 2024-04-26T21:23:03.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.087+0000 7f9b5ffff640 1 -- 172.21.15.129:0/3974868131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b60109f10 msgr2=0x7f9b6010a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:03.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.087+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/3974868131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b60109f10 0x7f9b6010a390 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009950 tx=0x7f9b5402a480 comp rx=0 tx=0).stop 2024-04-26T21:23:03.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.087+0000 7f9b5ffff640 1 -- 172.21.15.129:0/3974868131 shutdown_connections 2024-04-26T21:23:03.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.087+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/3974868131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b6010a8d0 0x7f9b6010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.087+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/3974868131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b60109f10 0x7f9b6010a390 secure :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009950 tx=0x7f9b5402a480 comp rx=0 tx=0).stop 2024-04-26T21:23:03.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.087+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/3974868131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b60073c10 0x7f9b60074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.087+0000 7f9b5ffff640 1 -- 172.21.15.129:0/3974868131 >> 172.21.15.129:0/3974868131 conn(0x7f9b6006f780 msgr2=0x7f9b60071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:03.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.090+0000 7f9b5ffff640 1 -- 172.21.15.129:0/3974868131 shutdown_connections 2024-04-26T21:23:03.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.091+0000 7f9b5ffff640 1 -- 172.21.15.129:0/3974868131 wait complete. 2024-04-26T21:23:03.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.092+0000 7f9b5ffff640 1 Processor -- start 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.092+0000 7f9b5ffff640 1 -- start start 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.092+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b60073c10 0x7f9b60119060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.092+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b6010a8d0 0x7f9b601195a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.092+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b6011a620 0x7f9b6011aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.092+0000 7f9b5ffff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60111530 con 0x7f9b6011a620 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.092+0000 7f9b5ffff640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601113b0 con 0x7f9b6010a8d0 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.092+0000 7f9b5ffff640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601116b0 con 0x7f9b60073c10 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5effd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b60073c10 0x7f9b60119060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5effd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b60073c10 0x7f9b60119060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:58168/0 (socket says 172.21.15.129:58168) 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5effd640 1 -- 172.21.15.129:0/794789574 learned_addr learned my addr 172.21.15.129:0/794789574 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b6011a620 0x7f9b6011aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5effd640 1 -- 172.21.15.129:0/794789574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b6010a8d0 msgr2=0x7f9b601195a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:23:03.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5effd640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b6010a8d0 0x7f9b601195a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5effd640 1 -- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b6011a620 msgr2=0x7f9b6011aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:03.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5effd640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b6011a620 0x7f9b6011aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5effd640 1 -- 172.21.15.129:0/794789574 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6011b090 con 0x7f9b60073c10 2024-04-26T21:23:03.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b6011a620 0x7f9b6011aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:23:03.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.093+0000 7f9b5effd640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b60073c10 0x7f9b60119060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000b480 tx=0x7f9b5000b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:03.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.132+0000 7f9b47fff640 1 -- 172.21.15.129:0/794789574 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50014070 con 0x7f9b60073c10 2024-04-26T21:23:03.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.132+0000 7f9b5ffff640 1 -- 172.21.15.129:0/794789574 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6011b380 con 0x7f9b60073c10 2024-04-26T21:23:03.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.132+0000 7f9b5ffff640 1 -- 172.21.15.129:0/794789574 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6007adf0 con 0x7f9b60073c10 2024-04-26T21:23:03.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.132+0000 7f9b47fff640 1 -- 172.21.15.129:0/794789574 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b50010d20 con 0x7f9b60073c10 2024-04-26T21:23:03.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.132+0000 7f9b47fff640 1 -- 172.21.15.129:0/794789574 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50017360 con 0x7f9b60073c10 2024-04-26T21:23:03.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.136+0000 7f9b68001740 1 Processor -- start 2024-04-26T21:23:03.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.138+0000 7f9b68001740 1 -- start start 2024-04-26T21:23:03.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.138+0000 7f9b68001740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba50834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.138+0000 7f9b68001740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 0x561ba5084700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.138+0000 7f9b68001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.138+0000 7f9b68001740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561ba4fe7d80 con 0x561ba5084c40 2024-04-26T21:23:03.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.138+0000 7f9b68001740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561ba4fa18b0 con 0x561ba50830b0 2024-04-26T21:23:03.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.138+0000 7f9b68001740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561ba4fe3d40 con 0x561ba5084280 2024-04-26T21:23:03.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.139+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba50834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.139+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba50834d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:55476/0 (socket says 172.21.15.129:55476) 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.139+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/1324461415 learned_addr learned my addr 172.21.15.129:0/1324461415 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.139+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.139+0000 7f9b5effd640 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 0x561ba5084700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.139+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/1324461415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 msgr2=0x561ba5084700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.139+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 0x561ba5084700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.139+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/1324461415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 msgr2=0x561ba508b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.140+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.140+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/1324461415 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba4f72a40 con 0x561ba50830b0 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.140+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.140+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba50834d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800c450 tx=0x7f9b5800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfde3e0df002902b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:03.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.141+0000 7f9b47fff640 1 -- 172.21.15.129:0/794789574 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9b50017070 con 0x7f9b60073c10 2024-04-26T21:23:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.147+0000 7f9b47fff640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b480a3e40 0x7f9b480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.152+0000 7f9b5effd640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b480a3e40 0x7f9b480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.153+0000 7f9b5effd640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b480a3e40 0x7f9b480a6300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9b500cd4d0 tx=0x7f9b500d4210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.154+0000 7f9b47fff640 1 -- 172.21.15.129:0/794789574 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9b500cfa80 con 0x7f9b60073c10 2024-04-26T21:23:03.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.172+0000 7f9b457fa640 1 -- 172.21.15.129:0/1324461415 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5801a070 con 0x561ba50830b0 2024-04-26T21:23:03.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.172+0000 7f9b457fa640 1 -- 172.21.15.129:0/1324461415 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b580078e0 con 0x561ba50830b0 2024-04-26T21:23:03.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.172+0000 7f9b457fa640 1 -- 172.21.15.129:0/1324461415 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58015800 con 0x561ba50830b0 2024-04-26T21:23:03.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.173+0000 7f9b68001740 1 -- 172.21.15.129:0/1324461415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 msgr2=0x561ba50834d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:03.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.173+0000 7f9b68001740 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba50834d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800c450 tx=0x7f9b5800c910 comp rx=0 tx=0).stop 2024-04-26T21:23:03.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.177+0000 7f9b68001740 1 -- 172.21.15.129:0/1324461415 shutdown_connections 2024-04-26T21:23:03.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.177+0000 7f9b68001740 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.177+0000 7f9b68001740 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 0x561ba5084700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.177+0000 7f9b68001740 1 --2- 172.21.15.129:0/1324461415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba50834d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.177+0000 7f9b68001740 1 -- 172.21.15.129:0/1324461415 >> 172.21.15.129:0/1324461415 conn(0x561ba507e5a0 msgr2=0x561ba507e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:03.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.183+0000 7f9b68001740 1 -- 172.21.15.129:0/1324461415 shutdown_connections 2024-04-26T21:23:03.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.183+0000 7f9b68001740 1 -- 172.21.15.129:0/1324461415 wait complete. 2024-04-26T21:23:03.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.183+0000 7f9b68001740 1 -- start start 2024-04-26T21:23:03.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b68001740 1 Processor -- start 2024-04-26T21:23:03.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b68001740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 0x561ba508d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b68001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084280 0x561ba508dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b68001740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084c40 0x561ba5094af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b68001740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561ba508bc50 con 0x561ba5084280 2024-04-26T21:23:03.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b68001740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561ba4f6ed10 con 0x561ba5084c40 2024-04-26T21:23:03.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b68001740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561ba508bdd0 con 0x561ba50830b0 2024-04-26T21:23:03.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b68001740 10 client.0 _ll_register_callbacks cb 0x561ba4f9b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:23:03.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084c40 0x561ba5094af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084c40 0x561ba5094af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:55484/0 (socket says 172.21.15.129:55484) 2024-04-26T21:23:03.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.184+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/1990131863 learned_addr learned my addr 172.21.15.129:0/1990131863 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:03.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.185+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 0x561ba508d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.185+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084280 0x561ba508dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.185+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/1990131863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 msgr2=0x561ba508d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:03.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.185+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 0x561ba508d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.185+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/1990131863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084280 msgr2=0x561ba508dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:03.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.185+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084280 0x561ba508dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.185+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/1990131863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba4dc37d0 con 0x561ba5084c40 2024-04-26T21:23:03.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.185+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084280 0x561ba508dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:23:03.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.185+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084c40 0x561ba5094af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009610 tx=0x7f9b580096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caab603b47cb626e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.202+0000 7f9b42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:23:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.202+0000 7f9b42ff5640 1 -- 172.21.15.129:0/1990131863 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5801a070 con 0x561ba5084c40 2024-04-26T21:23:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.202+0000 7f9b68001740 1 -- 172.21.15.129:0/1990131863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b580078e0 con 0x561ba5084c40 2024-04-26T21:23:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.202+0000 7f9b42ff5640 1 -- 172.21.15.129:0/1990131863 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9b580169d0 con 0x561ba5084c40 2024-04-26T21:23:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.202+0000 7f9b42ff5640 1 -- 172.21.15.129:0/1990131863 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9b58007710 con 0x561ba5084c40 2024-04-26T21:23:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.203+0000 7f9b68001740 10 client.25472 fetch_fsmap learned FSMap version 19 2024-04-26T21:23:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.203+0000 7f9b68001740 1 -- 172.21.15.129:0/1990131863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561ba4cf93f0 con 0x561ba5084c40 2024-04-26T21:23:03.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.205+0000 7f9b42ff5640 1 -- 172.21.15.129:0/1990131863 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b580078e0 con 0x561ba5084c40 2024-04-26T21:23:03.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.206+0000 7f9b42ff5640 1 -- 172.21.15.129:0/1990131863 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f9b5800ab90 con 0x561ba5084c40 2024-04-26T21:23:03.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.206+0000 7f9b68001740 10 client.25472 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:23:03.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.206+0000 7f9b68001740 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:23:03.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.206+0000 7f9b68001740 20 client.25472 populate_metadata read hostname 'smithi129' 2024-04-26T21:23:03.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.206+0000 7f9b68001740 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561ba5099380 0x561ba509b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.206+0000 7f9b68001740 4 client.25472 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:23:03.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.206+0000 7f9b68001740 1 -- 172.21.15.129:0/1990131863 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ba4ff5990 con 0x561ba5099380 2024-04-26T21:23:03.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.207+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561ba5099380 0x561ba509b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.207+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561ba5099380 0x561ba509b7a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1dedbd574ec8fdb server_cookie=fab4959060c00091 in_seq=0 out_seq=0 2024-04-26T21:23:03.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.207+0000 7f9b42ff5640 10 client.25472 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:23:03.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.215+0000 7f9b42ff5640 1 -- 172.21.15.129:0/1990131863 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ba4ff5990 con 0x561ba5099380 2024-04-26T21:23:03.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.215+0000 7f9b42ff5640 10 client.25472 handle_command_reply: tid=0 2024-04-26T21:23:03.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.215+0000 7f9b68001740 1 client.25472 shutdown 2024-04-26T21:23:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 -- 172.21.15.129:0/1990131863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084c40 msgr2=0x561ba5094af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084c40 0x561ba5094af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009610 tx=0x7f9b580096c0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 -- 172.21.15.129:0/1990131863 shutdown_connections 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561ba5099380 0x561ba509b7a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084c40 0x561ba5094af0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084280 0x561ba508dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 --2- 172.21.15.129:0/1990131863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 0x561ba508d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 -- 172.21.15.129:0/1990131863 >> 172.21.15.129:0/1990131863 conn(0x561ba507e5a0 msgr2=0x561ba507d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b42ff5640 0 client.25472 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 -- 172.21.15.129:0/1990131863 shutdown_connections 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.226+0000 7f9b68001740 1 -- 172.21.15.129:0/1990131863 wait complete. 2024-04-26T21:23:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.228+0000 7f9b68001740 20 client.25472 trim_cache size 0 max 16384 2024-04-26T21:23:03.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.456+0000 7f9b68001740 1 Processor -- start 2024-04-26T21:23:03.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.456+0000 7f9b68001740 1 -- start start 2024-04-26T21:23:03.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.461+0000 7f9b68001740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 0x561ba509b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.461+0000 7f9b68001740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084280 0x561ba5094ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.461+0000 7f9b68001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.461+0000 7f9b68001740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c00c4c0 con 0x561ba5084c40 2024-04-26T21:23:03.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.461+0000 7f9b68001740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c00ec90 con 0x561ba5084280 2024-04-26T21:23:03.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.461+0000 7f9b68001740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c00ca80 con 0x561ba50830b0 2024-04-26T21:23:03.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.462+0000 7f9b5e7fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084280 0x561ba5094ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.462+0000 7f9b5e7fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084280 0x561ba5094ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:55500/0 (socket says 172.21.15.129:55500) 2024-04-26T21:23:03.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.462+0000 7f9b5e7fc640 1 -- 172.21.15.129:0/3747219532 learned_addr learned my addr 172.21.15.129:0/3747219532 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:03.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.462+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/3747219532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.492+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/3747219532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 0x561ba509b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.492+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/3747219532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 msgr2=0x561ba509b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:03.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.492+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/3747219532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 0x561ba509b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.492+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/3747219532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084280 msgr2=0x561ba5094ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:03.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.492+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/3747219532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084280 0x561ba5094ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.492+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/3747219532 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba4cb90f0 con 0x561ba5084c40 2024-04-26T21:23:03.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.493+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/3747219532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508d570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9b580099c0 tx=0x7f9b5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2087e8d2f3f137 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:03.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.503+0000 7f9b44ff9640 1 -- 172.21.15.129:0/3747219532 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5801a070 con 0x561ba5084c40 2024-04-26T21:23:03.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.503+0000 7f9b44ff9640 1 -- 172.21.15.129:0/3747219532 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b5801b040 con 0x561ba5084c40 2024-04-26T21:23:03.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.503+0000 7f9b44ff9640 1 -- 172.21.15.129:0/3747219532 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b580239e0 con 0x561ba5084c40 2024-04-26T21:23:03.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.504+0000 7f9b68001740 1 -- 172.21.15.129:0/3747219532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 msgr2=0x561ba508d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:03.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.504+0000 7f9b68001740 1 --2- 172.21.15.129:0/3747219532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508d570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9b580099c0 tx=0x7f9b5800c910 comp rx=0 tx=0).stop 2024-04-26T21:23:03.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.506+0000 7f9b68001740 1 -- 172.21.15.129:0/3747219532 shutdown_connections 2024-04-26T21:23:03.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.506+0000 7f9b68001740 1 --2- 172.21.15.129:0/3747219532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba5084280 0x561ba5094ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.506+0000 7f9b68001740 1 --2- 172.21.15.129:0/3747219532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba508d570 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.506+0000 7f9b68001740 1 --2- 172.21.15.129:0/3747219532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba50830b0 0x561ba509b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.506+0000 7f9b68001740 1 -- 172.21.15.129:0/3747219532 >> 172.21.15.129:0/3747219532 conn(0x561ba507e5a0 msgr2=0x561ba507e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:03.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.510+0000 7f9b68001740 1 -- 172.21.15.129:0/3747219532 shutdown_connections 2024-04-26T21:23:03.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.510+0000 7f9b68001740 1 -- 172.21.15.129:0/3747219532 wait complete. 2024-04-26T21:23:03.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.510+0000 7f9b68001740 1 -- start start 2024-04-26T21:23:03.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.511+0000 7f9b68001740 1 Processor -- start 2024-04-26T21:23:03.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.511+0000 7f9b68001740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba5090cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.511+0000 7f9b68001740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 0x561ba50911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.512+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba5090cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.512+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba5090cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:55512/0 (socket says 172.21.15.129:55512) 2024-04-26T21:23:03.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.512+0000 7f9b5f7fe640 1 -- 172.21.15.129:0/3270927627 learned_addr learned my addr 172.21.15.129:0/3270927627 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:03.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.512+0000 7f9b68001740 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba5093eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.512+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 0x561ba50911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.512+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c00d150 con 0x561ba5084c40 2024-04-26T21:23:03.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.512+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c00e5c0 con 0x561ba50830b0 2024-04-26T21:23:03.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.512+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c00d820 con 0x561ba5084280 2024-04-26T21:23:03.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.522+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba5093eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.523+0000 7f9b5e7fc640 1 -- 172.21.15.129:0/3270927627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 msgr2=0x561ba5090cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:03.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.523+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba5090cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.523+0000 7f9b5e7fc640 1 -- 172.21.15.129:0/3270927627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 msgr2=0x561ba5093eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:03.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.523+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba5093eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.523+0000 7f9b5e7fc640 1 -- 172.21.15.129:0/3270927627 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba4fa84b0 con 0x561ba5084280 2024-04-26T21:23:03.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.523+0000 7f9b5e7fc640 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 0x561ba50911f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b5402aa70 tx=0x7f9b54005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f8a2e4633ab915 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:03.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.525+0000 7f9b68001740 10 client.0 _ll_register_callbacks cb 0x7f9b58018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:23:03.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.525+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b5801b040 con 0x561ba5084280 2024-04-26T21:23:03.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.536+0000 7f9b42ff5640 10 client.25402 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-26T21:23:03.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.536+0000 7f9b42ff5640 1 -- 172.21.15.129:0/3270927627 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54035040 con 0x561ba5084280 2024-04-26T21:23:03.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.536+0000 7f9b42ff5640 1 -- 172.21.15.129:0/3270927627 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9b54031b80 con 0x561ba5084280 2024-04-26T21:23:03.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.536+0000 7f9b42ff5640 1 -- 172.21.15.129:0/3270927627 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9b54031d80 con 0x561ba5084280 2024-04-26T21:23:03.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.536+0000 7f9b68001740 10 client.25402 fetch_fsmap learned FSMap version 19 2024-04-26T21:23:03.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.536+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({fsmap=19}) -- 0x561ba50fdcd0 con 0x561ba5084280 2024-04-26T21:23:03.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.541+0000 7f9b42ff5640 1 -- 172.21.15.129:0/3270927627 <== mon.1 v2:172.21.15.171:3300/0 4 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f9b5403a040 con 0x561ba5084280 2024-04-26T21:23:03.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.541+0000 7f9b68001740 10 client.25402 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:23:03.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.541+0000 7f9b68001740 10 client.25402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:23:03.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.541+0000 7f9b68001740 20 client.25402 populate_metadata read hostname 'smithi129' 2024-04-26T21:23:03.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.541+0000 7f9b68001740 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561ba50fe5c0 0x561ba5100a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:03.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.541+0000 7f9b68001740 4 client.25402 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:23:03.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.541+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9b5801b040 con 0x561ba50fe5c0 2024-04-26T21:23:03.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.541+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561ba50fe5c0 0x561ba5100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:03.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.542+0000 7f9b5f7fe640 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561ba50fe5c0 0x561ba5100a80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd964adf35c869e server_cookie=25e9803172190cff in_seq=0 out_seq=0 2024-04-26T21:23:03.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.543+0000 7f9b42ff5640 10 client.25402 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:23:03.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.553+0000 7f9b42ff5640 1 -- 172.21.15.129:0/3270927627 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x7f9b5801b040 con 0x561ba50fe5c0 2024-04-26T21:23:03.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.553+0000 7f9b42ff5640 10 client.25402 handle_command_reply: tid=0 2024-04-26T21:23:03.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.553+0000 7f9b68001740 1 client.25402 shutdown 2024-04-26T21:23:03.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 msgr2=0x561ba50911f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:03.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 0x561ba50911f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b5402aa70 tx=0x7f9b54005da0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 shutdown_connections 2024-04-26T21:23:03.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x561ba50fe5c0 0x561ba5100a80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ba5084c40 0x561ba5093eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ba5084280 0x561ba50911f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 --2- 172.21.15.129:0/3270927627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba50830b0 0x561ba5090cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 >> 172.21.15.129:0/3270927627 conn(0x561ba507e5a0 msgr2=0x561ba5079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:03.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b42ff5640 0 client.25402 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:23:03.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 shutdown_connections 2024-04-26T21:23:03.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 1 -- 172.21.15.129:0/3270927627 wait complete. 2024-04-26T21:23:03.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.554+0000 7f9b68001740 20 client.25402 trim_cache size 0 max 16384 2024-04-26T21:23:03.557 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:23:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (9908 inodes in the stack)", 2024-04-26T21:23:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:23:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "5846dc40-4197-47a8-90b0-6ec0e5d15dd0": { 2024-04-26T21:23:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:23:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "5846dc40-4197-47a8-90b0-6ec0e5d15dd0", 2024-04-26T21:23:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:23:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:23:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:23:03.558 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:23:03.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.576+0000 7f9b5ffff640 1 -- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b480a3e40 msgr2=0x7f9b480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:03.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.576+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b480a3e40 0x7f9b480a6300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9b500cd4d0 tx=0x7f9b500d4210 comp rx=0 tx=0).stop 2024-04-26T21:23:03.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.576+0000 7f9b5ffff640 1 -- 172.21.15.129:0/794789574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b60073c10 msgr2=0x7f9b60119060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:03.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.576+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b60073c10 0x7f9b60119060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000b480 tx=0x7f9b5000b940 comp rx=0 tx=0).stop 2024-04-26T21:23:03.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.579+0000 7f9b5ffff640 1 -- 172.21.15.129:0/794789574 shutdown_connections 2024-04-26T21:23:03.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.579+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f9b480a3e40 0x7f9b480a6300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.579+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b6010a8d0 0x7f9b601195a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.579+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b6011a620 0x7f9b6011aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.579+0000 7f9b5ffff640 1 --2- 172.21.15.129:0/794789574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b60073c10 0x7f9b60119060 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.579+0000 7f9b5ffff640 1 -- 172.21.15.129:0/794789574 >> 172.21.15.129:0/794789574 conn(0x7f9b6006f780 msgr2=0x7f9b6010d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.579+0000 7f9b5ffff640 1 -- 172.21.15.129:0/794789574 shutdown_connections 2024-04-26T21:23:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:03.580+0000 7f9b5ffff640 1 -- 172.21.15.129:0/794789574 wait complete. 2024-04-26T21:23:03.682 INFO:tasks.cephfs.filesystem:scrub status for tag:5846dc40-4197-47a8-90b0-6ec0e5d15dd0 - {'path': '/', 'tag': '5846dc40-4197-47a8-90b0-6ec0e5d15dd0', 'options': 'recursive,force'} 2024-04-26T21:23:03.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:03 smithi150 ceph-mon[32290]: pgmap v502: 97 pgs: 97 active+clean; 668 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 277 op/s 2024-04-26T21:23:03.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:03 smithi129 ceph-mon[29237]: pgmap v502: 97 pgs: 97 active+clean; 668 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 277 op/s 2024-04-26T21:23:03.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:03 smithi171 ceph-mon[33478]: pgmap v502: 97 pgs: 97 active+clean; 668 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 277 op/s 2024-04-26T21:23:05.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:05 smithi150 ceph-mon[32290]: pgmap v503: 97 pgs: 97 active+clean; 667 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-26T21:23:05.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:05 smithi129 ceph-mon[29237]: pgmap v503: 97 pgs: 97 active+clean; 667 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-26T21:23:05.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:05 smithi171 ceph-mon[33478]: pgmap v503: 97 pgs: 97 active+clean; 667 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-26T21:23:07.777 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:07 smithi129 ceph-mon[29237]: pgmap v504: 97 pgs: 97 active+clean; 669 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-04-26T21:23:07.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:07 smithi150 ceph-mon[32290]: pgmap v504: 97 pgs: 97 active+clean; 669 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-04-26T21:23:07.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:07 smithi171 ceph-mon[33478]: pgmap v504: 97 pgs: 97 active+clean; 669 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-04-26T21:23:09.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:09 smithi150 ceph-mon[32290]: pgmap v505: 97 pgs: 97 active+clean; 673 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-04-26T21:23:09.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:09 smithi129 ceph-mon[29237]: pgmap v505: 97 pgs: 97 active+clean; 673 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-04-26T21:23:09.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:09 smithi171 ceph-mon[33478]: pgmap v505: 97 pgs: 97 active+clean; 673 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-04-26T21:23:11.787 INFO:tasks.workunit.client.0.smithi129.stdout: 140 1463644 260 950545 190 769816 711 2024-04-26T21:23:11.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:11 smithi150 ceph-mon[32290]: pgmap v506: 97 pgs: 97 active+clean; 704 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 538 op/s 2024-04-26T21:23:11.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:11 smithi129 ceph-mon[29237]: pgmap v506: 97 pgs: 97 active+clean; 704 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 538 op/s 2024-04-26T21:23:11.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:11 smithi171 ceph-mon[33478]: pgmap v506: 97 pgs: 97 active+clean; 704 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 538 op/s 2024-04-26T21:23:13.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:13 smithi150 ceph-mon[32290]: pgmap v507: 97 pgs: 97 active+clean; 718 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 682 op/s 2024-04-26T21:23:13.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:13 smithi129 ceph-mon[29237]: pgmap v507: 97 pgs: 97 active+clean; 718 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 682 op/s 2024-04-26T21:23:13.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:13 smithi171 ceph-mon[33478]: pgmap v507: 97 pgs: 97 active+clean; 718 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 682 op/s 2024-04-26T21:23:15.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:15 smithi171 ceph-mon[33478]: pgmap v508: 97 pgs: 97 active+clean; 715 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-04-26T21:23:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:15 smithi150 ceph-mon[32290]: pgmap v508: 97 pgs: 97 active+clean; 715 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-04-26T21:23:15.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:15 smithi129 ceph-mon[29237]: pgmap v508: 97 pgs: 97 active+clean; 715 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-04-26T21:23:17.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:17 smithi150 ceph-mon[32290]: pgmap v509: 97 pgs: 97 active+clean; 718 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.7 MiB/s wr, 825 op/s 2024-04-26T21:23:17.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:17 smithi129 ceph-mon[29237]: pgmap v509: 97 pgs: 97 active+clean; 718 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.7 MiB/s wr, 825 op/s 2024-04-26T21:23:17.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:17 smithi171 ceph-mon[33478]: pgmap v509: 97 pgs: 97 active+clean; 718 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.7 MiB/s wr, 825 op/s 2024-04-26T21:23:19.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:19 smithi150 ceph-mon[32290]: pgmap v510: 97 pgs: 97 active+clean; 719 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 889 op/s 2024-04-26T21:23:19.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:19 smithi129 ceph-mon[29237]: pgmap v510: 97 pgs: 97 active+clean; 719 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 889 op/s 2024-04-26T21:23:19.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:19 smithi171 ceph-mon[33478]: pgmap v510: 97 pgs: 97 active+clean; 719 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 889 op/s 2024-04-26T21:23:21.787 INFO:tasks.workunit.client.0.smithi129.stdout: 148 2191927 501 1421171 515 1152406 1524 2024-04-26T21:23:21.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:21 smithi150 ceph-mon[32290]: pgmap v511: 97 pgs: 97 active+clean; 721 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 767 op/s 2024-04-26T21:23:21.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:21 smithi129 ceph-mon[29237]: pgmap v511: 97 pgs: 97 active+clean; 721 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 767 op/s 2024-04-26T21:23:21.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:21 smithi171 ceph-mon[33478]: pgmap v511: 97 pgs: 97 active+clean; 721 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 767 op/s 2024-04-26T21:23:23.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:23 smithi150 ceph-mon[32290]: pgmap v512: 97 pgs: 97 active+clean; 724 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 602 op/s 2024-04-26T21:23:23.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:23 smithi129 ceph-mon[29237]: pgmap v512: 97 pgs: 97 active+clean; 724 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 602 op/s 2024-04-26T21:23:23.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:23 smithi171 ceph-mon[33478]: pgmap v512: 97 pgs: 97 active+clean; 724 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 602 op/s 2024-04-26T21:23:25.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:25 smithi150 ceph-mon[32290]: pgmap v513: 97 pgs: 97 active+clean; 725 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 466 op/s 2024-04-26T21:23:25.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:25 smithi129 ceph-mon[29237]: pgmap v513: 97 pgs: 97 active+clean; 725 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 466 op/s 2024-04-26T21:23:25.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:25 smithi171 ceph-mon[33478]: pgmap v513: 97 pgs: 97 active+clean; 725 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 466 op/s 2024-04-26T21:23:27.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:27 smithi150 ceph-mon[32290]: pgmap v514: 97 pgs: 97 active+clean; 720 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 409 op/s 2024-04-26T21:23:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:27 smithi129 ceph-mon[29237]: pgmap v514: 97 pgs: 97 active+clean; 720 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 409 op/s 2024-04-26T21:23:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:27 smithi171 ceph-mon[33478]: pgmap v514: 97 pgs: 97 active+clean; 720 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 409 op/s 2024-04-26T21:23:29.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:29 smithi150 ceph-mon[32290]: pgmap v515: 97 pgs: 97 active+clean; 699 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 346 op/s 2024-04-26T21:23:29.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:29 smithi129 ceph-mon[29237]: pgmap v515: 97 pgs: 97 active+clean; 699 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 346 op/s 2024-04-26T21:23:29.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:29 smithi171 ceph-mon[33478]: pgmap v515: 97 pgs: 97 active+clean; 699 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 346 op/s 2024-04-26T21:23:30.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:31.787 INFO:tasks.workunit.client.0.smithi129.stdout: 155 2388917 335 1550865 289 1297275 1740 2024-04-26T21:23:31.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:31 smithi150 ceph-mon[32290]: pgmap v516: 97 pgs: 97 active+clean; 700 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2024-04-26T21:23:31.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:31 smithi129 ceph-mon[29237]: pgmap v516: 97 pgs: 97 active+clean; 700 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2024-04-26T21:23:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:31 smithi171 ceph-mon[33478]: pgmap v516: 97 pgs: 97 active+clean; 700 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2024-04-26T21:23:33.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:23:33.683 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:23:33.787 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:33 smithi129 ceph-mon[29237]: pgmap v517: 97 pgs: 97 active+clean; 714 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-04-26T21:23:33.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:33 smithi150 ceph-mon[32290]: pgmap v517: 97 pgs: 97 active+clean; 714 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-04-26T21:23:33.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.947+0000 7f79657d5640 1 -- 172.21.15.129:0/2493682437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7960109f10 msgr2=0x7f796010a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:33.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.947+0000 7f79657d5640 1 --2- 172.21.15.129:0/2493682437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7960109f10 0x7f796010a390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7958009ca0 tx=0x7f795802a8f0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.952+0000 7f79657d5640 1 -- 172.21.15.129:0/2493682437 shutdown_connections 2024-04-26T21:23:33.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.952+0000 7f79657d5640 1 --2- 172.21.15.129:0/2493682437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7960109f10 0x7f796010a390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.952+0000 7f79657d5640 1 --2- 172.21.15.129:0/2493682437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f796010a8d0 0x7f796010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.952+0000 7f79657d5640 1 --2- 172.21.15.129:0/2493682437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7960073c10 0x7f7960074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.952+0000 7f79657d5640 1 -- 172.21.15.129:0/2493682437 >> 172.21.15.129:0/2493682437 conn(0x7f796006f780 msgr2=0x7f7960071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:33.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.955+0000 7f79657d5640 1 -- 172.21.15.129:0/2493682437 shutdown_connections 2024-04-26T21:23:33.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.963+0000 7f79657d5640 1 -- 172.21.15.129:0/2493682437 wait complete. 2024-04-26T21:23:33.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.964+0000 7f79657d5640 1 Processor -- start 2024-04-26T21:23:33.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.964+0000 7f79657d5640 1 -- start start 2024-04-26T21:23:33.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f79657d5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7960073c10 0x7f79601a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:33.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f79657d5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7960109f10 0x7f79601a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:33.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f79657d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f796010a8d0 0x7f7960113070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:33.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f79657d5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f796011d170 con 0x7f7960073c10 2024-04-26T21:23:33.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f79657d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f796011cff0 con 0x7f796010a8d0 2024-04-26T21:23:33.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f79657d5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f796011d2f0 con 0x7f7960109f10 2024-04-26T21:23:33.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f795f7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7960109f10 0x7f79601a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f795f7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7960109f10 0x7f79601a7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:57278/0 (socket says 172.21.15.129:57278) 2024-04-26T21:23:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f795f7fe640 1 -- 172.21.15.129:0/2039727449 learned_addr learned my addr 172.21.15.129:0/2039727449 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f7964fd4640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f796010a8d0 0x7f7960113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f795f7fe640 1 -- 172.21.15.129:0/2039727449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f796010a8d0 msgr2=0x7f7960113070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f795f7fe640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f796010a8d0 0x7f7960113070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.965+0000 7f795f7fe640 1 -- 172.21.15.129:0/2039727449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7960073c10 msgr2=0x7f79601a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.966+0000 7f795f7fe640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7960073c10 0x7f79601a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.966+0000 7f795f7fe640 1 -- 172.21.15.129:0/2039727449 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79601136e0 con 0x7f7960109f10 2024-04-26T21:23:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.966+0000 7f795f7fe640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7960109f10 0x7f79601a7db0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7958031b50 tx=0x7f7958031930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:33.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.977+0000 7f795d7fa640 1 -- 172.21.15.129:0/2039727449 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958004040 con 0x7f7960109f10 2024-04-26T21:23:33.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.977+0000 7f795d7fa640 1 -- 172.21.15.129:0/2039727449 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795802d070 con 0x7f7960109f10 2024-04-26T21:23:33.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.977+0000 7f79657d5640 1 -- 172.21.15.129:0/2039727449 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7960113970 con 0x7f7960109f10 2024-04-26T21:23:33.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.977+0000 7f795d7fa640 1 -- 172.21.15.129:0/2039727449 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795802fc90 con 0x7f7960109f10 2024-04-26T21:23:33.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.977+0000 7f79657d5640 1 -- 172.21.15.129:0/2039727449 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7960113f40 con 0x7f7960109f10 2024-04-26T21:23:33.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.981+0000 7f795d7fa640 1 -- 172.21.15.129:0/2039727449 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f79580041e0 con 0x7f7960109f10 2024-04-26T21:23:33.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.982+0000 7f796982f740 1 Processor -- start 2024-04-26T21:23:33.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.982+0000 7f796982f740 1 -- start start 2024-04-26T21:23:33.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.982+0000 7f796982f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efda4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:33.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.982+0000 7f796982f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efdb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:33.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.982+0000 7f796982f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:33.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.982+0000 7f796982f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a4ef3ed80 con 0x562a4efdbc40 2024-04-26T21:23:33.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.982+0000 7f796982f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562a4eef88b0 con 0x562a4efdb280 2024-04-26T21:23:33.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.982+0000 7f796982f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562a4ef3ad40 con 0x562a4efda0b0 2024-04-26T21:23:33.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.983+0000 7f795d7fa640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79480a3dc0 0x7f79480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:33.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.983+0000 7f795d7fa640 1 -- 172.21.15.129:0/2039727449 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7958113b50 con 0x7f7960109f10 2024-04-26T21:23:33.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.984+0000 7f7964fd4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efda4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:33.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.984+0000 7f7964fd4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efda4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:57286/0 (socket says 172.21.15.129:57286) 2024-04-26T21:23:33.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.984+0000 7f7964fd4640 1 -- 172.21.15.129:0/913965892 learned_addr learned my addr 172.21.15.129:0/913965892 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:33.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.985+0000 7f795f7fe640 1 --2- 172.21.15.129:0/913965892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:33.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.985+0000 7f7964fd4640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79480a3dc0 0x7f79480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:33.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.986+0000 7f795ffff640 1 --2- 172.21.15.129:0/913965892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efdb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:33.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.986+0000 7f7964fd4640 1 -- 172.21.15.129:0/913965892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 msgr2=0x562a4efdb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:33.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.986+0000 7f7964fd4640 1 --2- 172.21.15.129:0/913965892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efdb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.986+0000 7f7964fd4640 1 -- 172.21.15.129:0/913965892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 msgr2=0x562a4efe24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:33.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.986+0000 7f7964fd4640 1 --2- 172.21.15.129:0/913965892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.986+0000 7f7964fd4640 1 -- 172.21.15.129:0/913965892 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4eec9a40 con 0x562a4efda0b0 2024-04-26T21:23:33.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.990+0000 7f7964fd4640 1 --2- 172.21.15.129:0/913965892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efda4d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79500116d0 tx=0x7f7950011b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bc4bda9f3ee9857 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:33.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:33 smithi171 ceph-mon[33478]: pgmap v517: 97 pgs: 97 active+clean; 714 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-04-26T21:23:33.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.990+0000 7f7964fd4640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79480a3dc0 0x7f79480a6280 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f795001b000 tx=0x7f795001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:33.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.991+0000 7f79467fc640 1 -- 172.21.15.129:0/913965892 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795001c9d0 con 0x562a4efda0b0 2024-04-26T21:23:33.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.991+0000 7f79467fc640 1 -- 172.21.15.129:0/913965892 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7950020030 con 0x562a4efda0b0 2024-04-26T21:23:33.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.993+0000 7f796982f740 1 -- 172.21.15.129:0/913965892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 msgr2=0x562a4efda4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:33.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.993+0000 7f796982f740 1 --2- 172.21.15.129:0/913965892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efda4d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79500116d0 tx=0x7f7950011b90 comp rx=0 tx=0).stop 2024-04-26T21:23:33.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.995+0000 7f796982f740 1 -- 172.21.15.129:0/913965892 shutdown_connections 2024-04-26T21:23:33.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.995+0000 7f796982f740 1 --2- 172.21.15.129:0/913965892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efdb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.995+0000 7f796982f740 1 --2- 172.21.15.129:0/913965892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.995+0000 7f796982f740 1 --2- 172.21.15.129:0/913965892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efda4d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:33.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:33.995+0000 7f796982f740 1 -- 172.21.15.129:0/913965892 >> 172.21.15.129:0/913965892 conn(0x562a4efd55a0 msgr2=0x562a4efd59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:34.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.001+0000 7f796982f740 1 -- 172.21.15.129:0/913965892 shutdown_connections 2024-04-26T21:23:34.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.002+0000 7f796982f740 1 -- 172.21.15.129:0/913965892 wait complete. 2024-04-26T21:23:34.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.003+0000 7f796982f740 1 -- start start 2024-04-26T21:23:34.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.003+0000 7f796982f740 1 Processor -- start 2024-04-26T21:23:34.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.004+0000 7f796982f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.004+0000 7f796982f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.004+0000 7f796982f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.005+0000 7f796982f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a4efe2c40 con 0x562a4efdbc40 2024-04-26T21:23:34.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.005+0000 7f796982f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562a4eec5d10 con 0x562a4efdb280 2024-04-26T21:23:34.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.005+0000 7f796982f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562a4efe2dc0 con 0x562a4efda0b0 2024-04-26T21:23:34.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.005+0000 7f796982f740 10 client.0 _ll_register_callbacks cb 0x562a4eef2210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:23:34.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.005+0000 7f795ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.005+0000 7f795ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:57302/0 (socket says 172.21.15.129:57302) 2024-04-26T21:23:34.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.005+0000 7f795ffff640 1 -- 172.21.15.129:0/3841247603 learned_addr learned my addr 172.21.15.129:0/3841247603 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:34.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.005+0000 7f7964fd4640 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.006+0000 7f7964fd4640 1 -- 172.21.15.129:0/3841247603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 msgr2=0x562a4efe44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:34.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.006+0000 7f7964fd4640 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.006+0000 7f7964fd4640 1 -- 172.21.15.129:0/3841247603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 msgr2=0x562a4efe4a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:23:34.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.006+0000 7f7964fd4640 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe4a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.006+0000 7f7964fd4640 1 -- 172.21.15.129:0/3841247603 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4ed1a7d0 con 0x562a4efdbc40 2024-04-26T21:23:34.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.007+0000 7f7964fd4640 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7950005850 tx=0x7f795000ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3368b90174a5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:34.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.007+0000 7f793f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:23:34.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.009+0000 7f793f7fe640 1 -- 172.21.15.129:0/3841247603 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950029040 con 0x562a4efdbc40 2024-04-26T21:23:34.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.009+0000 7f796982f740 1 -- 172.21.15.129:0/3841247603 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7950020030 con 0x562a4efdbc40 2024-04-26T21:23:34.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.009+0000 7f793f7fe640 1 -- 172.21.15.129:0/3841247603 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7950025670 con 0x562a4efdbc40 2024-04-26T21:23:34.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.018+0000 7f793f7fe640 1 -- 172.21.15.129:0/3841247603 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f795001c7d0 con 0x562a4efdbc40 2024-04-26T21:23:34.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.020+0000 7f796982f740 10 client.15891 fetch_fsmap learned FSMap version 19 2024-04-26T21:23:34.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.020+0000 7f796982f740 1 -- 172.21.15.129:0/3841247603 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562a4ec503f0 con 0x562a4efdbc40 2024-04-26T21:23:34.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.021+0000 7f793f7fe640 1 -- 172.21.15.129:0/3841247603 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7950020030 con 0x562a4efdbc40 2024-04-26T21:23:34.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.021+0000 7f793f7fe640 1 -- 172.21.15.129:0/3841247603 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f7950004680 con 0x562a4efdbc40 2024-04-26T21:23:34.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.021+0000 7f796982f740 10 client.15891 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:23:34.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.021+0000 7f796982f740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:23:34.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.021+0000 7f796982f740 20 client.15891 populate_metadata read hostname 'smithi129' 2024-04-26T21:23:34.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.021+0000 7f796982f740 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x562a4eff0430 0x562a4eff2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.021+0000 7f796982f740 4 client.15891 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:23:34.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.021+0000 7f796982f740 1 -- 172.21.15.129:0/3841247603 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a4ef4c990 con 0x562a4eff0430 2024-04-26T21:23:34.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.021+0000 7f795ffff640 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x562a4eff0430 0x562a4eff2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.031+0000 7f795ffff640 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x562a4eff0430 0x562a4eff2880 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3334e122dcb1feb server_cookie=f5ce9ff449982873 in_seq=0 out_seq=0 2024-04-26T21:23:34.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.031+0000 7f793f7fe640 10 client.15891 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:23:34.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.035+0000 7f793f7fe640 1 -- 172.21.15.129:0/3841247603 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a4ef4c990 con 0x562a4eff0430 2024-04-26T21:23:34.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.035+0000 7f793f7fe640 10 client.15891 handle_command_reply: tid=0 2024-04-26T21:23:34.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.035+0000 7f796982f740 1 client.15891 shutdown 2024-04-26T21:23:34.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.036+0000 7f796982f740 1 -- 172.21.15.129:0/3841247603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 msgr2=0x562a4efeba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:34.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.036+0000 7f796982f740 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7950005850 tx=0x7f795000ac00 comp rx=0 tx=0).stop 2024-04-26T21:23:34.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.041+0000 7f796982f740 1 -- 172.21.15.129:0/3841247603 shutdown_connections 2024-04-26T21:23:34.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.041+0000 7f796982f740 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x562a4eff0430 0x562a4eff2880 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.041+0000 7f796982f740 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.041+0000 7f796982f740 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.041+0000 7f796982f740 1 --2- 172.21.15.129:0/3841247603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.041+0000 7f796982f740 1 -- 172.21.15.129:0/3841247603 >> 172.21.15.129:0/3841247603 conn(0x562a4efd55a0 msgr2=0x562a4efd46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:34.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.041+0000 7f793f7fe640 0 client.15891 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:23:34.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.044+0000 7f796982f740 1 -- 172.21.15.129:0/3841247603 shutdown_connections 2024-04-26T21:23:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.047+0000 7f796982f740 1 -- 172.21.15.129:0/3841247603 wait complete. 2024-04-26T21:23:34.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.048+0000 7f796982f740 20 client.15891 trim_cache size 0 max 16384 2024-04-26T21:23:34.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.100+0000 7f796982f740 1 Processor -- start 2024-04-26T21:23:34.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.100+0000 7f796982f740 1 -- start start 2024-04-26T21:23:34.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.100+0000 7f796982f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.101+0000 7f796982f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.101+0000 7f796982f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.101+0000 7f796982f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f79601ae3f0 con 0x562a4efdbc40 2024-04-26T21:23:34.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.101+0000 7f796982f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79601b0c70 con 0x562a4efdb280 2024-04-26T21:23:34.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.101+0000 7f796982f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f79601aea60 con 0x562a4efda0b0 2024-04-26T21:23:34.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.101+0000 7f795ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.101+0000 7f795ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:57306/0 (socket says 172.21.15.129:57306) 2024-04-26T21:23:34.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.101+0000 7f795ffff640 1 -- 172.21.15.129:0/2452750916 learned_addr learned my addr 172.21.15.129:0/2452750916 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:34.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.101+0000 7f7964fd4640 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.105+0000 7f795ffff640 1 -- 172.21.15.129:0/2452750916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 msgr2=0x562a4efe4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:34.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.105+0000 7f795f7fe640 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.106+0000 7f795ffff640 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.106+0000 7f795ffff640 1 -- 172.21.15.129:0/2452750916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 msgr2=0x562a4efeba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:34.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.106+0000 7f795ffff640 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.106+0000 7f795ffff640 1 -- 172.21.15.129:0/2452750916 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4ec100f0 con 0x562a4efda0b0 2024-04-26T21:23:34.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.106+0000 7f795f7fe640 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:23:34.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.106+0000 7f7964fd4640 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:23:34.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.106+0000 7f795ffff640 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f795400b570 tx=0x7f795400b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cde3b9b50be8f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:34.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.107+0000 7f7945ffb640 1 -- 172.21.15.129:0/2452750916 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954015630 con 0x562a4efda0b0 2024-04-26T21:23:34.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.107+0000 7f7945ffb640 1 -- 172.21.15.129:0/2452750916 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795401e030 con 0x562a4efda0b0 2024-04-26T21:23:34.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.107+0000 7f7945ffb640 1 -- 172.21.15.129:0/2452750916 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954022bf0 con 0x562a4efda0b0 2024-04-26T21:23:34.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.108+0000 7f796982f740 1 -- 172.21.15.129:0/2452750916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 msgr2=0x562a4efe44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:34.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.108+0000 7f796982f740 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f795400b570 tx=0x7f795400b800 comp rx=0 tx=0).stop 2024-04-26T21:23:34.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.108+0000 7f796982f740 1 -- 172.21.15.129:0/2452750916 shutdown_connections 2024-04-26T21:23:34.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.108+0000 7f796982f740 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.108+0000 7f796982f740 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efeba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.108+0000 7f796982f740 1 --2- 172.21.15.129:0/2452750916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4efda0b0 0x562a4efe44e0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f795400b570 tx=0x7f795400b800 comp rx=0 tx=0).stop 2024-04-26T21:23:34.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.108+0000 7f796982f740 1 -- 172.21.15.129:0/2452750916 >> 172.21.15.129:0/2452750916 conn(0x562a4efd55a0 msgr2=0x562a4efdedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:34.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.114+0000 7f796982f740 1 -- 172.21.15.129:0/2452750916 shutdown_connections 2024-04-26T21:23:34.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.116+0000 7f796982f740 1 -- 172.21.15.129:0/2452750916 wait complete. 2024-04-26T21:23:34.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.116+0000 7f796982f740 1 -- start start 2024-04-26T21:23:34.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.116+0000 7f796982f740 1 Processor -- start 2024-04-26T21:23:34.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f796982f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f796982f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f796982f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4f0529e0 0x562a4f054dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f796982f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f79601af130 con 0x562a4efdbc40 2024-04-26T21:23:34.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f796982f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79601b05a0 con 0x562a4efdb280 2024-04-26T21:23:34.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f796982f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f79601af800 con 0x562a4f0529e0 2024-04-26T21:23:34.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f796982f740 10 client.0 _ll_register_callbacks cb 0x7f79500114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:23:34.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f795f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f795f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44192/0 (socket says 172.21.15.129:44192) 2024-04-26T21:23:34.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.117+0000 7f795f7fe640 1 -- 172.21.15.129:0/1223783016 learned_addr learned my addr 172.21.15.129:0/1223783016 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:23:34.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.119+0000 7f795ffff640 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.119+0000 7f7964fd4640 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4f0529e0 0x562a4f054dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.120+0000 7f795f7fe640 1 -- 172.21.15.129:0/1223783016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4f0529e0 msgr2=0x562a4f054dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:34.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.121+0000 7f795f7fe640 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4f0529e0 0x562a4f054dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.121+0000 7f795f7fe640 1 -- 172.21.15.129:0/1223783016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 msgr2=0x562a4efe88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:23:34.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.121+0000 7f795f7fe640 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.121+0000 7f795f7fe640 1 -- 172.21.15.129:0/1223783016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4eeff4b0 con 0x562a4efdbc40 2024-04-26T21:23:34.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.121+0000 7f795f7fe640 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe8e30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7958006010 tx=0x7f7958009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1ec6485460936 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:34.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.121+0000 7f795ffff640 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:23:34.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.121+0000 7f7964fd4640 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4f0529e0 0x562a4f054dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:23:34.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.122+0000 7f793f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:23:34.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.122+0000 7f793f7fe640 1 -- 172.21.15.129:0/1223783016 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580f8070 con 0x562a4efdbc40 2024-04-26T21:23:34.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.122+0000 7f793f7fe640 1 -- 172.21.15.129:0/1223783016 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f79580f9270 con 0x562a4efdbc40 2024-04-26T21:23:34.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.122+0000 7f796982f740 1 -- 172.21.15.129:0/1223783016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f795401e030 con 0x562a4efdbc40 2024-04-26T21:23:34.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.122+0000 7f793f7fe640 1 -- 172.21.15.129:0/1223783016 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f79580e4180 con 0x562a4efdbc40 2024-04-26T21:23:34.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.125+0000 7f796982f740 10 client.15897 fetch_fsmap learned FSMap version 19 2024-04-26T21:23:34.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.125+0000 7f796982f740 1 -- 172.21.15.129:0/1223783016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562a4f056430 con 0x562a4efdbc40 2024-04-26T21:23:34.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.126+0000 7f793f7fe640 1 -- 172.21.15.129:0/1223783016 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795401e030 con 0x562a4efdbc40 2024-04-26T21:23:34.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.127+0000 7f793f7fe640 1 -- 172.21.15.129:0/1223783016 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f79580e0030 con 0x562a4efdbc40 2024-04-26T21:23:34.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.128+0000 7f796982f740 10 client.15897 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:23:34.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.128+0000 7f796982f740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:23:34.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.128+0000 7f796982f740 20 client.15897 populate_metadata read hostname 'smithi129' 2024-04-26T21:23:34.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.128+0000 7f796982f740 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x562a4f056d40 0x562a4f059160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:34.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.128+0000 7f796982f740 4 client.15897 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:23:34.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.128+0000 7f796982f740 1 -- 172.21.15.129:0/1223783016 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562a4ef46800 con 0x562a4f056d40 2024-04-26T21:23:34.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.128+0000 7f795ffff640 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x562a4f056d40 0x562a4f059160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:34.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.136+0000 7f795ffff640 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x562a4f056d40 0x562a4f059160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234b32482e780850 server_cookie=c5134437d37d792e in_seq=0 out_seq=0 2024-04-26T21:23:34.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.136+0000 7f793f7fe640 10 client.15897 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:23:34.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.143+0000 7f793f7fe640 1 -- 172.21.15.129:0/1223783016 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x562a4ef46800 con 0x562a4f056d40 2024-04-26T21:23:34.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.143+0000 7f793f7fe640 10 client.15897 handle_command_reply: tid=0 2024-04-26T21:23:34.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.144+0000 7f796982f740 1 client.15897 shutdown 2024-04-26T21:23:34.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.149+0000 7f796982f740 1 -- 172.21.15.129:0/1223783016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 msgr2=0x562a4efe8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:34.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.149+0000 7f796982f740 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe8e30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7958006010 tx=0x7f7958009990 comp rx=0 tx=0).stop 2024-04-26T21:23:34.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.149+0000 7f796982f740 1 -- 172.21.15.129:0/1223783016 shutdown_connections 2024-04-26T21:23:34.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.149+0000 7f796982f740 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x562a4f056d40 0x562a4f059160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.149+0000 7f796982f740 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a4f0529e0 0x562a4f054dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.149+0000 7f796982f740 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a4efdbc40 0x562a4efe8e30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.149+0000 7f796982f740 1 --2- 172.21.15.129:0/1223783016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a4efdb280 0x562a4efe88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.149+0000 7f796982f740 1 -- 172.21.15.129:0/1223783016 >> 172.21.15.129:0/1223783016 conn(0x562a4efd55a0 msgr2=0x562a4efdedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:34.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.150+0000 7f793f7fe640 0 client.15897 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:23:34.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.152+0000 7f796982f740 1 -- 172.21.15.129:0/1223783016 shutdown_connections 2024-04-26T21:23:34.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.152+0000 7f796982f740 1 -- 172.21.15.129:0/1223783016 wait complete. 2024-04-26T21:23:34.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.154+0000 7f796982f740 20 client.15897 trim_cache size 0 max 16384 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (5960 inodes in the stack)", 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout: "5846dc40-4197-47a8-90b0-6ec0e5d15dd0": { 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "5846dc40-4197-47a8-90b0-6ec0e5d15dd0", 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:23:34.161 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:23:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.163+0000 7f79657d5640 1 -- 172.21.15.129:0/2039727449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79480a3dc0 msgr2=0x7f79480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.163+0000 7f79657d5640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79480a3dc0 0x7f79480a6280 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f795001b000 tx=0x7f795001d000 comp rx=0 tx=0).stop 2024-04-26T21:23:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.164+0000 7f79657d5640 1 -- 172.21.15.129:0/2039727449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7960109f10 msgr2=0x7f79601a7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.164+0000 7f79657d5640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7960109f10 0x7f79601a7db0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7958031b50 tx=0x7f7958031930 comp rx=0 tx=0).stop 2024-04-26T21:23:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.165+0000 7f79657d5640 1 -- 172.21.15.129:0/2039727449 shutdown_connections 2024-04-26T21:23:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.165+0000 7f79657d5640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f79480a3dc0 0x7f79480a6280 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.165+0000 7f79657d5640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f796010a8d0 0x7f7960113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.165+0000 7f79657d5640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7960109f10 0x7f79601a7db0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.165+0000 7f79657d5640 1 --2- 172.21.15.129:0/2039727449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7960073c10 0x7f79601a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.166+0000 7f79657d5640 1 -- 172.21.15.129:0/2039727449 >> 172.21.15.129:0/2039727449 conn(0x7f796006f780 msgr2=0x7f7960071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:34.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.170+0000 7f79657d5640 1 -- 172.21.15.129:0/2039727449 shutdown_connections 2024-04-26T21:23:34.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:23:34.171+0000 7f79657d5640 1 -- 172.21.15.129:0/2039727449 wait complete. 2024-04-26T21:23:34.203 INFO:tasks.cephfs.filesystem:scrub status for tag:5846dc40-4197-47a8-90b0-6ec0e5d15dd0 - {'path': '/', 'tag': '5846dc40-4197-47a8-90b0-6ec0e5d15dd0', 'options': 'recursive,force'} 2024-04-26T21:23:34.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:23:34.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:34.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:34.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:23:34.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:34.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:34.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:23:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:35 smithi150 ceph-mon[32290]: pgmap v518: 97 pgs: 97 active+clean; 733 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 373 op/s 2024-04-26T21:23:35.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:35 smithi129 ceph-mon[29237]: pgmap v518: 97 pgs: 97 active+clean; 733 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 373 op/s 2024-04-26T21:23:35.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:35 smithi171 ceph-mon[33478]: pgmap v518: 97 pgs: 97 active+clean; 733 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 373 op/s 2024-04-26T21:23:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:37 smithi129 ceph-mon[29237]: pgmap v519: 97 pgs: 97 active+clean; 757 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 528 op/s 2024-04-26T21:23:37.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:37 smithi150 ceph-mon[32290]: pgmap v519: 97 pgs: 97 active+clean; 757 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 528 op/s 2024-04-26T21:23:37.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:37 smithi171 ceph-mon[33478]: pgmap v519: 97 pgs: 97 active+clean; 757 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 528 op/s 2024-04-26T21:23:39.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:39 smithi150 ceph-mon[32290]: pgmap v520: 97 pgs: 97 active+clean; 775 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 744 op/s 2024-04-26T21:23:39.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:39 smithi129 ceph-mon[29237]: pgmap v520: 97 pgs: 97 active+clean; 775 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 744 op/s 2024-04-26T21:23:39.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:39 smithi171 ceph-mon[33478]: pgmap v520: 97 pgs: 97 active+clean; 775 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 744 op/s 2024-04-26T21:23:41.787 INFO:tasks.workunit.client.0.smithi129.stdout: 156 1232739 236 807084 196 683435 840 2024-04-26T21:23:41.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:41 smithi150 ceph-mon[32290]: pgmap v521: 97 pgs: 97 active+clean; 765 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 857 op/s 2024-04-26T21:23:41.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:41 smithi129 ceph-mon[29237]: pgmap v521: 97 pgs: 97 active+clean; 765 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 857 op/s 2024-04-26T21:23:41.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:41 smithi171 ceph-mon[33478]: pgmap v521: 97 pgs: 97 active+clean; 765 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 857 op/s 2024-04-26T21:23:43.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:43 smithi150 ceph-mon[32290]: pgmap v522: 97 pgs: 97 active+clean; 748 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.6 MiB/s wr, 958 op/s 2024-04-26T21:23:43.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:43 smithi129 ceph-mon[29237]: pgmap v522: 97 pgs: 97 active+clean; 748 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.6 MiB/s wr, 958 op/s 2024-04-26T21:23:43.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:43 smithi171 ceph-mon[33478]: pgmap v522: 97 pgs: 97 active+clean; 748 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.6 MiB/s wr, 958 op/s 2024-04-26T21:23:45.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:45 smithi171 ceph-mon[33478]: pgmap v523: 97 pgs: 97 active+clean; 742 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 912 op/s 2024-04-26T21:23:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:45 smithi150 ceph-mon[32290]: pgmap v523: 97 pgs: 97 active+clean; 742 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 912 op/s 2024-04-26T21:23:45.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:45 smithi129 ceph-mon[29237]: pgmap v523: 97 pgs: 97 active+clean; 742 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 912 op/s 2024-04-26T21:23:47.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:47 smithi150 ceph-mon[32290]: pgmap v524: 97 pgs: 97 active+clean; 748 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 802 op/s 2024-04-26T21:23:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:47 smithi129 ceph-mon[29237]: pgmap v524: 97 pgs: 97 active+clean; 748 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 802 op/s 2024-04-26T21:23:47.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:47 smithi171 ceph-mon[33478]: pgmap v524: 97 pgs: 97 active+clean; 748 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 802 op/s 2024-04-26T21:23:49.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:49 smithi150 ceph-mon[32290]: pgmap v525: 97 pgs: 97 active+clean; 747 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 698 op/s 2024-04-26T21:23:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:49 smithi129 ceph-mon[29237]: pgmap v525: 97 pgs: 97 active+clean; 747 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 698 op/s 2024-04-26T21:23:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:49 smithi171 ceph-mon[33478]: pgmap v525: 97 pgs: 97 active+clean; 747 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 698 op/s 2024-04-26T21:23:51.787 INFO:tasks.workunit.client.0.smithi129.stdout: 160 2305906 291 1524590 309 1312822 1790 2024-04-26T21:23:51.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:51 smithi150 ceph-mon[32290]: pgmap v526: 97 pgs: 97 active+clean; 745 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 518 op/s 2024-04-26T21:23:51.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:51 smithi129 ceph-mon[29237]: pgmap v526: 97 pgs: 97 active+clean; 745 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 518 op/s 2024-04-26T21:23:51.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:51 smithi171 ceph-mon[33478]: pgmap v526: 97 pgs: 97 active+clean; 745 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 518 op/s 2024-04-26T21:23:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:53 smithi150 ceph-mon[32290]: pgmap v527: 97 pgs: 97 active+clean; 746 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-04-26T21:23:53.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:53 smithi129 ceph-mon[29237]: pgmap v527: 97 pgs: 97 active+clean; 746 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-04-26T21:23:53.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:53 smithi171 ceph-mon[33478]: pgmap v527: 97 pgs: 97 active+clean; 746 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-04-26T21:23:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:55 smithi150 ceph-mon[32290]: pgmap v528: 97 pgs: 97 active+clean; 749 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 340 op/s 2024-04-26T21:23:55.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:55 smithi129 ceph-mon[29237]: pgmap v528: 97 pgs: 97 active+clean; 749 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 340 op/s 2024-04-26T21:23:55.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:55 smithi171 ceph-mon[33478]: pgmap v528: 97 pgs: 97 active+clean; 749 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 340 op/s 2024-04-26T21:23:57.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:57 smithi150 ceph-mon[32290]: pgmap v529: 97 pgs: 97 active+clean; 756 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-04-26T21:23:57.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:57 smithi129 ceph-mon[29237]: pgmap v529: 97 pgs: 97 active+clean; 756 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-04-26T21:23:57.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:57 smithi171 ceph-mon[33478]: pgmap v529: 97 pgs: 97 active+clean; 756 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-04-26T21:23:59.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:23:59 smithi150 ceph-mon[32290]: pgmap v530: 97 pgs: 97 active+clean; 743 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 311 op/s 2024-04-26T21:23:59.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:23:59 smithi129 ceph-mon[29237]: pgmap v530: 97 pgs: 97 active+clean; 743 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 311 op/s 2024-04-26T21:23:59.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:23:59 smithi171 ceph-mon[33478]: pgmap v530: 97 pgs: 97 active+clean; 743 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 311 op/s 2024-04-26T21:24:01.787 INFO:tasks.workunit.client.0.smithi129.stdout: 163 1732706 259 1154233 150 1020570 1064 2024-04-26T21:24:02.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:01 smithi150 ceph-mon[32290]: pgmap v531: 97 pgs: 97 active+clean; 757 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 446 op/s 2024-04-26T21:24:02.219 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:01 smithi129 ceph-mon[29237]: pgmap v531: 97 pgs: 97 active+clean; 757 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 446 op/s 2024-04-26T21:24:02.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:01 smithi171 ceph-mon[33478]: pgmap v531: 97 pgs: 97 active+clean; 757 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 446 op/s 2024-04-26T21:24:03.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:03 smithi150 ceph-mon[32290]: pgmap v532: 97 pgs: 97 active+clean; 784 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-04-26T21:24:03.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:03 smithi129 ceph-mon[29237]: pgmap v532: 97 pgs: 97 active+clean; 784 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-04-26T21:24:03.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:03 smithi171 ceph-mon[33478]: pgmap v532: 97 pgs: 97 active+clean; 784 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-04-26T21:24:04.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:24:04.204 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:24:04.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.399+0000 7ff814fda640 1 -- 172.21.15.129:0/1060787561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810076910 msgr2=0x7ff810074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:04.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.399+0000 7ff814fda640 1 --2- 172.21.15.129:0/1060787561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810076910 0x7ff810074d60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff800009900 tx=0x7ff80002a480 comp rx=0 tx=0).stop 2024-04-26T21:24:04.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.399+0000 7ff814fda640 1 -- 172.21.15.129:0/1060787561 shutdown_connections 2024-04-26T21:24:04.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.399+0000 7ff814fda640 1 --2- 172.21.15.129:0/1060787561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff81010a810 0x7ff81010ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.399+0000 7ff814fda640 1 --2- 172.21.15.129:0/1060787561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810075330 0x7ff810075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.399+0000 7ff814fda640 1 --2- 172.21.15.129:0/1060787561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810076910 0x7ff810074d60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.399+0000 7ff814fda640 1 -- 172.21.15.129:0/1060787561 >> 172.21.15.129:0/1060787561 conn(0x7ff8100fdf10 msgr2=0x7ff810100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:04.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.399+0000 7ff814fda640 1 -- 172.21.15.129:0/1060787561 shutdown_connections 2024-04-26T21:24:04.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.399+0000 7ff814fda640 1 -- 172.21.15.129:0/1060787561 wait complete. 2024-04-26T21:24:04.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff814fda640 1 Processor -- start 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff814fda640 1 -- start start 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff814fda640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810075330 0x7ff81019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff814fda640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810076910 0x7ff81019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff814fda640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff81010a810 0x7ff8101a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff814fda640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff810110680 con 0x7ff810075330 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff814fda640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff810110500 con 0x7ff81010a810 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff814fda640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff810110800 con 0x7ff810076910 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff80e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810075330 0x7ff81019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff81010a810 0x7ff8101a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff80e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810075330 0x7ff81019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56672/0 (socket says 172.21.15.129:56672) 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff80dd9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810076910 0x7ff81019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.400+0000 7ff80e59c640 1 -- 172.21.15.129:0/20273462 learned_addr learned my addr 172.21.15.129:0/20273462 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff80e59c640 1 -- 172.21.15.129:0/20273462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810076910 msgr2=0x7ff81019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:04.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff80e59c640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810076910 0x7ff81019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff80e59c640 1 -- 172.21.15.129:0/20273462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff81010a810 msgr2=0x7ff8101a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff80e59c640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff81010a810 0x7ff8101a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff80e59c640 1 -- 172.21.15.129:0/20273462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8101a29f0 con 0x7ff810075330 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff81010a810 0x7ff8101a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff80dd9b640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810076910 0x7ff81019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff80e59c640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810075330 0x7ff81019ae20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff800005940 tx=0x7ff800031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff7ff7fe640 1 -- 172.21.15.129:0/20273462 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80003f030 con 0x7ff810075330 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff814fda640 1 -- 172.21.15.129:0/20273462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8101a2c80 con 0x7ff810075330 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff7ff7fe640 1 -- 172.21.15.129:0/20273462 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff800003aa0 con 0x7ff810075330 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff7ff7fe640 1 -- 172.21.15.129:0/20273462 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff800035070 con 0x7ff810075330 2024-04-26T21:24:04.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.401+0000 7ff814fda640 1 -- 172.21.15.129:0/20273462 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8101a31c0 con 0x7ff810075330 2024-04-26T21:24:04.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.404+0000 7ff7ff7fe640 1 -- 172.21.15.129:0/20273462 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff800042030 con 0x7ff810075330 2024-04-26T21:24:04.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.404+0000 7ff817dc7740 1 Processor -- start 2024-04-26T21:24:04.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.404+0000 7ff817dc7740 1 -- start start 2024-04-26T21:24:04.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.404+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.404+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ea7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63eae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff817dc7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ed63e0ad80 con 0x55ed63ea70b0 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff817dc7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ed63dc48b0 con 0x55ed63ea5ee0 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff817dc7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ed63e06d40 con 0x55ed63ea7a70 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:58116/0 (socket says 172.21.15.129:58116) 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80ed9d640 1 -- 172.21.15.129:0/1597299812 learned_addr learned my addr 172.21.15.129:0/1597299812 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80e59c640 1 --2- 172.21.15.129:0/1597299812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ea7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80dd9b640 1 --2- 172.21.15.129:0/1597299812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63eae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80ed9d640 1 -- 172.21.15.129:0/1597299812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 msgr2=0x55ed63eae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:04.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/1597299812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63eae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80ed9d640 1 -- 172.21.15.129:0/1597299812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 msgr2=0x55ed63ea7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/1597299812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ea7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80ed9d640 1 -- 172.21.15.129:0/1597299812 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed63d95a40 con 0x55ed63ea5ee0 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff7ff7fe640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff7f80a3c40 0x7ff7f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80e59c640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff7f80a3c40 0x7ff7f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.405+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/1597299812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea6300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8080117e0 tx=0x7ff808011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bdc1b03af733963 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff7fcff9640 1 -- 172.21.15.129:0/1597299812 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808011310 con 0x55ed63ea5ee0 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff7ff7fe640 1 -- 172.21.15.129:0/20273462 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff8000f0b30 con 0x7ff810075330 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff7fcff9640 1 -- 172.21.15.129:0/1597299812 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff808003e20 con 0x55ed63ea5ee0 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff7fcff9640 1 -- 172.21.15.129:0/1597299812 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808004490 con 0x55ed63ea5ee0 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff817dc7740 1 -- 172.21.15.129:0/1597299812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 msgr2=0x55ed63ea6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff80e59c640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff7f80a3c40 0x7ff7f80a6100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8000b9720 tx=0x7ff8000f2040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1597299812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea6300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8080117e0 tx=0x7ff808011ca0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff817dc7740 1 -- 172.21.15.129:0/1597299812 shutdown_connections 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1597299812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63eae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1597299812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ea7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1597299812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea6300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.406+0000 7ff817dc7740 1 -- 172.21.15.129:0/1597299812 >> 172.21.15.129:0/1597299812 conn(0x55ed63ea13d0 msgr2=0x55ed63ea17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 -- 172.21.15.129:0/1597299812 shutdown_connections 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 -- 172.21.15.129:0/1597299812 wait complete. 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 -- start start 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 Processor -- start 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63eb8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63eb9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63ec0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ed63eaea80 con 0x55ed63ea70b0 2024-04-26T21:24:04.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ed63d91d10 con 0x55ed63ea5ee0 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ed63eaec00 con 0x55ed63ea7a70 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff817dc7740 10 client.0 _ll_register_callbacks cb 0x55ed63dbe210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63eb8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63eb8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:58122/0 (socket says 172.21.15.129:58122) 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff80ed9d640 1 -- 172.21.15.129:0/1143047100 learned_addr learned my addr 172.21.15.129:0/1143047100 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.407+0000 7ff80dd9b640 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63eb9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63ec0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff80dd9b640 1 -- 172.21.15.129:0/1143047100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 msgr2=0x55ed63ec0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff80dd9b640 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63ec0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff80dd9b640 1 -- 172.21.15.129:0/1143047100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 msgr2=0x55ed63eb8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff80dd9b640 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63eb8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff80dd9b640 1 -- 172.21.15.129:0/1143047100 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed63be67d0 con 0x55ed63ea70b0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff80dd9b640 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63eb9110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff81019c2e0 tx=0x7ff80400f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfbf30da437ee99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff7f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/1143047100 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80400fdb0 con 0x55ed63ea70b0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/1143047100 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff804005020 con 0x55ed63ea70b0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff817dc7740 1 -- 172.21.15.129:0/1143047100 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff808003e20 con 0x55ed63ea70b0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/1143047100 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff804005220 con 0x55ed63ea70b0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff817dc7740 10 client.15909 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.408+0000 7ff817dc7740 1 -- 172.21.15.129:0/1143047100 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ed63b1c3f0 con 0x55ed63ea70b0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/1143047100 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff804004c50 con 0x55ed63ea70b0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/1143047100 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ff8040246a0 con 0x55ed63ea70b0 2024-04-26T21:24:04.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff817dc7740 10 client.15909 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:04.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff817dc7740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:04.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff817dc7740 20 client.15909 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:04.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55ed63ec4650 0x55ed63ec6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff817dc7740 4 client.15909 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:24:04.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff817dc7740 1 -- 172.21.15.129:0/1143047100 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff808003e20 con 0x55ed63ec4650 2024-04-26T21:24:04.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55ed63ec4650 0x55ed63ec6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff7f5ffb640 10 client.15909 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:04.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.409+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55ed63ec4650 0x55ed63ec6ab0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0e1fc54c5a0ca7 server_cookie=4b563c1fe7b31520 in_seq=0 out_seq=0 2024-04-26T21:24:04.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/1143047100 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff808003e20 con 0x55ed63ec4650 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff7f5ffb640 10 client.15909 handle_command_reply: tid=0 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 client.15909 shutdown 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 -- 172.21.15.129:0/1143047100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 msgr2=0x55ed63eb9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63eb9110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff81019c2e0 tx=0x7ff80400f420 comp rx=0 tx=0).stop 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 -- 172.21.15.129:0/1143047100 shutdown_connections 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55ed63ec4650 0x55ed63ec6ab0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63ec0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63eb9110 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff7f5ffb640 0 client.15909 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 --2- 172.21.15.129:0/1143047100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63eb8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 -- 172.21.15.129:0/1143047100 >> 172.21.15.129:0/1143047100 conn(0x55ed63ea13d0 msgr2=0x55ed63ea0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:04.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 -- 172.21.15.129:0/1143047100 shutdown_connections 2024-04-26T21:24:04.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 1 -- 172.21.15.129:0/1143047100 wait complete. 2024-04-26T21:24:04.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.413+0000 7ff817dc7740 20 client.15909 trim_cache size 0 max 16384 2024-04-26T21:24:04.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff817dc7740 1 Processor -- start 2024-04-26T21:24:04.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff817dc7740 1 -- start start 2024-04-26T21:24:04.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea70b0 0x55ed63ec0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea7a70 0x55ed63ea45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff817dc7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101a5bc0 con 0x55ed63ea7a70 2024-04-26T21:24:04.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff817dc7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101a8420 con 0x55ed63ea70b0 2024-04-26T21:24:04.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff817dc7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101a6210 con 0x55ed63ea5ee0 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea7a70 0x55ed63ea45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea70b0 0x55ed63ec0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea7a70 0x55ed63ea45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56700/0 (socket says 172.21.15.129:56700) 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 -- 172.21.15.129:0/2078634947 learned_addr learned my addr 172.21.15.129:0/2078634947 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/2078634947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 -- 172.21.15.129:0/2078634947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea5ee0 msgr2=0x55ed63ea40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/2078634947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 -- 172.21.15.129:0/2078634947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea70b0 msgr2=0x55ed63ec0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/2078634947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea70b0 0x55ed63ec0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 -- 172.21.15.129:0/2078634947 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed63adc0f0 con 0x55ed63ea7a70 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.432+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/2078634947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea7a70 0x55ed63ea45f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff808006010 tx=0x7ff808005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cff135aa8d7769 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.433+0000 7ff7f7fff640 1 -- 172.21.15.129:0/2078634947 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808025040 con 0x55ed63ea7a70 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.433+0000 7ff7f7fff640 1 -- 172.21.15.129:0/2078634947 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80800fe50 con 0x55ed63ea7a70 2024-04-26T21:24:04.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.433+0000 7ff7f7fff640 1 -- 172.21.15.129:0/2078634947 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80802f7d0 con 0x55ed63ea7a70 2024-04-26T21:24:04.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 -- 172.21.15.129:0/2078634947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea7a70 msgr2=0x55ed63ea45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:04.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 --2- 172.21.15.129:0/2078634947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea7a70 0x55ed63ea45f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff808006010 tx=0x7ff808005bd0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 -- 172.21.15.129:0/2078634947 shutdown_connections 2024-04-26T21:24:04.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 --2- 172.21.15.129:0/2078634947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea70b0 0x55ed63ec0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 --2- 172.21.15.129:0/2078634947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea7a70 0x55ed63ea45f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 --2- 172.21.15.129:0/2078634947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea5ee0 0x55ed63ea40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 -- 172.21.15.129:0/2078634947 >> 172.21.15.129:0/2078634947 conn(0x55ed63ea13d0 msgr2=0x55ed63e9c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:04.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 -- 172.21.15.129:0/2078634947 shutdown_connections 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 -- 172.21.15.129:0/2078634947 wait complete. 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 -- start start 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 Processor -- start 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ebd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ebded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63ebee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101a68e0 con 0x55ed63ea70b0 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101a7d50 con 0x55ed63ea5ee0 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101a6fb0 con 0x55ed63ea7a70 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff80dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ebded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff80dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ebded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56708/0 (socket says 172.21.15.129:56708) 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff80dd9b640 1 -- 172.21.15.129:0/3311703837 learned_addr learned my addr 172.21.15.129:0/3311703837 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ebd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.434+0000 7ff817dc7740 10 client.0 _ll_register_callbacks cb 0x7ff80400ab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:04.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63ebee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff80dd9b640 1 -- 172.21.15.129:0/3311703837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 msgr2=0x55ed63ebee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:04.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff80dd9b640 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63ebee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff80dd9b640 1 -- 172.21.15.129:0/3311703837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 msgr2=0x55ed63ebd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:04.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff80dd9b640 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ebd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff80dd9b640 1 -- 172.21.15.129:0/3311703837 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed63dcb4b0 con 0x55ed63ea70b0 2024-04-26T21:24:04.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63ebee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:24:04.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff80dd9b640 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ebded0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff804024a90 tx=0x7ff804005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570db6f42e976012 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:04.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff7f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/3311703837 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80402a040 con 0x55ed63ea70b0 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/3311703837 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff804024470 con 0x55ed63ea70b0 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff817dc7740 1 -- 172.21.15.129:0/3311703837 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff80800fe50 con 0x55ed63ea70b0 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/3311703837 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff804024670 con 0x55ed63ea70b0 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff817dc7740 10 client.15915 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff817dc7740 1 -- 172.21.15.129:0/3311703837 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ed63f2abf0 con 0x55ed63ea70b0 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/3311703837 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80800fe50 con 0x55ed63ea70b0 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/3311703837 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ff804038420 con 0x55ed63ea70b0 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff817dc7740 10 client.15915 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff817dc7740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.435+0000 7ff817dc7740 20 client.15915 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.436+0000 7ff817dc7740 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55ed63f2b4a0 0x55ed63f2d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.436+0000 7ff817dc7740 4 client.15915 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:24:04.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.436+0000 7ff817dc7740 1 -- 172.21.15.129:0/3311703837 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed63e18990 con 0x55ed63f2b4a0 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.436+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55ed63f2b4a0 0x55ed63f2d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.436+0000 7ff7f5ffb640 10 client.15915 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.436+0000 7ff80ed9d640 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55ed63f2b4a0 0x55ed63f2d8c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c6263ba4d8cfeb server_cookie=81ba98ecfc33cc1f in_seq=0 out_seq=0 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.438+0000 7ff7f5ffb640 1 -- 172.21.15.129:0/3311703837 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55ed63e18990 con 0x55ed63f2b4a0 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.438+0000 7ff7f5ffb640 10 client.15915 handle_command_reply: tid=0 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.438+0000 7ff817dc7740 1 client.15915 shutdown 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.438+0000 7ff817dc7740 1 -- 172.21.15.129:0/3311703837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 msgr2=0x55ed63ebded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.438+0000 7ff817dc7740 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ebded0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff804024a90 tx=0x7ff804005730 comp rx=0 tx=0).stop 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.438+0000 7ff817dc7740 1 -- 172.21.15.129:0/3311703837 shutdown_connections 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff817dc7740 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55ed63f2b4a0 0x55ed63f2d8c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff817dc7740 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed63ea7a70 0x55ed63ebee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff817dc7740 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ed63ea70b0 0x55ed63ebded0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff817dc7740 1 --2- 172.21.15.129:0/3311703837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed63ea5ee0 0x55ed63ebd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff7f5ffb640 0 client.15915 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:04.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff817dc7740 1 -- 172.21.15.129:0/3311703837 >> 172.21.15.129:0/3311703837 conn(0x55ed63ea13d0 msgr2=0x55ed63e9fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:04.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff817dc7740 1 -- 172.21.15.129:0/3311703837 shutdown_connections 2024-04-26T21:24:04.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff817dc7740 1 -- 172.21.15.129:0/3311703837 wait complete. 2024-04-26T21:24:04.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff817dc7740 20 client.15915 trim_cache size 0 max 16384 2024-04-26T21:24:04.445 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:24:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (1100 inodes in the stack)", 2024-04-26T21:24:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:24:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: "5846dc40-4197-47a8-90b0-6ec0e5d15dd0": { 2024-04-26T21:24:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:24:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "5846dc40-4197-47a8-90b0-6ec0e5d15dd0", 2024-04-26T21:24:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:24:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:24:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:24:04.446 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff814fda640 1 -- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff7f80a3c40 msgr2=0x7ff7f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff814fda640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff7f80a3c40 0x7ff7f80a6100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8000b9720 tx=0x7ff8000f2040 comp rx=0 tx=0).stop 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.439+0000 7ff814fda640 1 -- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810075330 msgr2=0x7ff81019ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.440+0000 7ff814fda640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810075330 0x7ff81019ae20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff800005940 tx=0x7ff800031a50 comp rx=0 tx=0).stop 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.440+0000 7ff814fda640 1 -- 172.21.15.129:0/20273462 shutdown_connections 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.440+0000 7ff814fda640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff7f80a3c40 0x7ff7f80a6100 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.440+0000 7ff814fda640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff81010a810 0x7ff8101a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.440+0000 7ff814fda640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810076910 0x7ff81019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.440+0000 7ff814fda640 1 --2- 172.21.15.129:0/20273462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff810075330 0x7ff81019ae20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.440+0000 7ff814fda640 1 -- 172.21.15.129:0/20273462 >> 172.21.15.129:0/20273462 conn(0x7ff8100fdf10 msgr2=0x7ff8101067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.440+0000 7ff814fda640 1 -- 172.21.15.129:0/20273462 shutdown_connections 2024-04-26T21:24:04.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:04.440+0000 7ff814fda640 1 -- 172.21.15.129:0/20273462 wait complete. 2024-04-26T21:24:04.452 INFO:tasks.cephfs.filesystem:scrub status for tag:5846dc40-4197-47a8-90b0-6ec0e5d15dd0 - {'path': '/', 'tag': '5846dc40-4197-47a8-90b0-6ec0e5d15dd0', 'options': 'recursive,force'} 2024-04-26T21:24:05.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:05 smithi150 ceph-mon[32290]: pgmap v533: 97 pgs: 97 active+clean; 792 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 723 op/s 2024-04-26T21:24:05.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:05 smithi129 ceph-mon[29237]: pgmap v533: 97 pgs: 97 active+clean; 792 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 723 op/s 2024-04-26T21:24:05.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:05 smithi171 ceph-mon[33478]: pgmap v533: 97 pgs: 97 active+clean; 792 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 723 op/s 2024-04-26T21:24:07.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:07 smithi129 ceph-mon[29237]: pgmap v534: 97 pgs: 97 active+clean; 798 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 813 op/s 2024-04-26T21:24:07.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:07 smithi150 ceph-mon[32290]: pgmap v534: 97 pgs: 97 active+clean; 798 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 813 op/s 2024-04-26T21:24:07.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:07 smithi171 ceph-mon[33478]: pgmap v534: 97 pgs: 97 active+clean; 798 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 813 op/s 2024-04-26T21:24:09.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:09 smithi150 ceph-mon[32290]: pgmap v535: 97 pgs: 97 active+clean; 801 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 951 op/s 2024-04-26T21:24:09.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:09 smithi129 ceph-mon[29237]: pgmap v535: 97 pgs: 97 active+clean; 801 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 951 op/s 2024-04-26T21:24:09.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:09 smithi171 ceph-mon[33478]: pgmap v535: 97 pgs: 97 active+clean; 801 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 951 op/s 2024-04-26T21:24:11.787 INFO:tasks.workunit.client.0.smithi129.stdout: 167 1654527 391 1091404 344 967335 1063 2024-04-26T21:24:11.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:11 smithi150 ceph-mon[32290]: pgmap v536: 97 pgs: 97 active+clean; 796 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 1.03k op/s 2024-04-26T21:24:11.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:11 smithi129 ceph-mon[29237]: pgmap v536: 97 pgs: 97 active+clean; 796 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 1.03k op/s 2024-04-26T21:24:11.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:11 smithi171 ceph-mon[33478]: pgmap v536: 97 pgs: 97 active+clean; 796 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 1.03k op/s 2024-04-26T21:24:13.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:13 smithi150 ceph-mon[32290]: pgmap v537: 97 pgs: 97 active+clean; 796 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 891 op/s 2024-04-26T21:24:13.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:13 smithi129 ceph-mon[29237]: pgmap v537: 97 pgs: 97 active+clean; 796 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 891 op/s 2024-04-26T21:24:13.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:13 smithi171 ceph-mon[33478]: pgmap v537: 97 pgs: 97 active+clean; 796 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 891 op/s 2024-04-26T21:24:15.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:15 smithi171 ceph-mon[33478]: pgmap v538: 97 pgs: 97 active+clean; 791 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 643 op/s 2024-04-26T21:24:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:15 smithi150 ceph-mon[32290]: pgmap v538: 97 pgs: 97 active+clean; 791 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 643 op/s 2024-04-26T21:24:15.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:15 smithi129 ceph-mon[29237]: pgmap v538: 97 pgs: 97 active+clean; 791 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 643 op/s 2024-04-26T21:24:17.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:17 smithi150 ceph-mon[32290]: pgmap v539: 97 pgs: 97 active+clean; 776 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 559 op/s 2024-04-26T21:24:17.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:17 smithi129 ceph-mon[29237]: pgmap v539: 97 pgs: 97 active+clean; 776 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 559 op/s 2024-04-26T21:24:17.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:17 smithi171 ceph-mon[33478]: pgmap v539: 97 pgs: 97 active+clean; 776 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 559 op/s 2024-04-26T21:24:19.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:19 smithi150 ceph-mon[32290]: pgmap v540: 97 pgs: 97 active+clean; 769 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 482 op/s 2024-04-26T21:24:19.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:19 smithi129 ceph-mon[29237]: pgmap v540: 97 pgs: 97 active+clean; 769 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 482 op/s 2024-04-26T21:24:19.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:19 smithi171 ceph-mon[33478]: pgmap v540: 97 pgs: 97 active+clean; 769 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 482 op/s 2024-04-26T21:24:21.787 INFO:tasks.workunit.client.0.smithi129.stdout: 171 1948358 257 1306403 337 1116478 1206 2024-04-26T21:24:21.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:21 smithi150 ceph-mon[32290]: pgmap v541: 97 pgs: 97 active+clean; 786 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 459 op/s 2024-04-26T21:24:21.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:21 smithi129 ceph-mon[29237]: pgmap v541: 97 pgs: 97 active+clean; 786 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 459 op/s 2024-04-26T21:24:21.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:21 smithi171 ceph-mon[33478]: pgmap v541: 97 pgs: 97 active+clean; 786 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 459 op/s 2024-04-26T21:24:23.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:23 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:24:23.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:23 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:24:23.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:23 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:24:24.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:24 smithi129 ceph-mon[29237]: pgmap v542: 97 pgs: 97 active+clean; 814 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-04-26T21:24:24.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:24 smithi171 ceph-mon[33478]: pgmap v542: 97 pgs: 97 active+clean; 814 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-04-26T21:24:24.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:24 smithi150 ceph-mon[32290]: pgmap v542: 97 pgs: 97 active+clean; 814 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-04-26T21:24:25.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:25 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:24:25.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:25 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:24:25.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:25 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:24:26.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:26 smithi129 ceph-mon[29237]: pgmap v543: 97 pgs: 97 active+clean; 820 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.7 MiB/s wr, 572 op/s 2024-04-26T21:24:26.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:26 smithi171 ceph-mon[33478]: pgmap v543: 97 pgs: 97 active+clean; 820 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.7 MiB/s wr, 572 op/s 2024-04-26T21:24:26.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:26 smithi150 ceph-mon[32290]: pgmap v543: 97 pgs: 97 active+clean; 820 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.7 MiB/s wr, 572 op/s 2024-04-26T21:24:27.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:27 smithi150 ceph-mon[32290]: pgmap v544: 97 pgs: 97 active+clean; 826 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-04-26T21:24:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:27 smithi129 ceph-mon[29237]: pgmap v544: 97 pgs: 97 active+clean; 826 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-04-26T21:24:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:27 smithi171 ceph-mon[33478]: pgmap v544: 97 pgs: 97 active+clean; 826 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-04-26T21:24:29.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:29 smithi150 ceph-mon[32290]: pgmap v545: 97 pgs: 97 active+clean; 825 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 712 op/s 2024-04-26T21:24:29.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:29 smithi129 ceph-mon[29237]: pgmap v545: 97 pgs: 97 active+clean; 825 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 712 op/s 2024-04-26T21:24:29.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:29 smithi171 ceph-mon[33478]: pgmap v545: 97 pgs: 97 active+clean; 825 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 712 op/s 2024-04-26T21:24:31.787 INFO:tasks.workunit.client.0.smithi129.stdout: 177 1988646 548 1351350 477 1131487 1195 2024-04-26T21:24:31.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:31 smithi150 ceph-mon[32290]: pgmap v546: 97 pgs: 97 active+clean; 827 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 756 op/s 2024-04-26T21:24:31.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:31 smithi129 ceph-mon[29237]: pgmap v546: 97 pgs: 97 active+clean; 827 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 756 op/s 2024-04-26T21:24:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:31 smithi171 ceph-mon[33478]: pgmap v546: 97 pgs: 97 active+clean; 827 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 756 op/s 2024-04-26T21:24:33.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:33 smithi150 ceph-mon[32290]: pgmap v547: 97 pgs: 97 active+clean; 817 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 701 op/s 2024-04-26T21:24:33.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:33 smithi129 ceph-mon[29237]: pgmap v547: 97 pgs: 97 active+clean; 817 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 701 op/s 2024-04-26T21:24:33.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:33 smithi171 ceph-mon[33478]: pgmap v547: 97 pgs: 97 active+clean; 817 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 701 op/s 2024-04-26T21:24:34.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:24:34.453 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:24:34.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.558+0000 7f33acd14640 1 -- 172.21.15.129:0/3369479207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a8103450 msgr2=0x7f33a8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.558+0000 7f33acd14640 1 --2- 172.21.15.129:0/3369479207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a8103450 0x7f33a8105840 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f339c009900 tx=0x7f339c02a410 comp rx=0 tx=0).stop 2024-04-26T21:24:34.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.559+0000 7f33acd14640 1 -- 172.21.15.129:0/3369479207 shutdown_connections 2024-04-26T21:24:34.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.559+0000 7f33acd14640 1 --2- 172.21.15.129:0/3369479207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a81086b0 0x7f33a810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.559+0000 7f33acd14640 1 --2- 172.21.15.129:0/3369479207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a8105d80 0x7f33a8108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.559+0000 7f33acd14640 1 --2- 172.21.15.129:0/3369479207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a8103450 0x7f33a8105840 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.559+0000 7f33acd14640 1 -- 172.21.15.129:0/3369479207 >> 172.21.15.129:0/3369479207 conn(0x7f33a80fcde0 msgr2=0x7f33a80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:34.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.559+0000 7f33acd14640 1 -- 172.21.15.129:0/3369479207 shutdown_connections 2024-04-26T21:24:34.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.559+0000 7f33acd14640 1 -- 172.21.15.129:0/3369479207 wait complete. 2024-04-26T21:24:34.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.559+0000 7f33acd14640 1 Processor -- start 2024-04-26T21:24:34.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.559+0000 7f33acd14640 1 -- start start 2024-04-26T21:24:34.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33acd14640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a8103450 0x7f33a819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33acd14640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a8105d80 0x7f33a819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33acd14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a81086b0 0x7f33a81a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33acd14640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33a810e4c0 con 0x7f33a81086b0 2024-04-26T21:24:34.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33acd14640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33a810e340 con 0x7f33a8105d80 2024-04-26T21:24:34.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33acd14640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f33a810e640 con 0x7f33a8103450 2024-04-26T21:24:34.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a81086b0 0x7f33a81a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a5d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a8105d80 0x7f33a819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a659c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a8103450 0x7f33a819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a81086b0 0x7f33a81a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50568/0 (socket says 172.21.15.129:50568) 2024-04-26T21:24:34.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a6d9d640 1 -- 172.21.15.129:0/3222557408 learned_addr learned my addr 172.21.15.129:0/3222557408 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:34.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a5d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a8105d80 0x7f33a819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:56944/0 (socket says 172.21.15.129:56944) 2024-04-26T21:24:34.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a6d9d640 1 -- 172.21.15.129:0/3222557408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a8103450 msgr2=0x7f33a819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a6d9d640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a8103450 0x7f33a819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a6d9d640 1 -- 172.21.15.129:0/3222557408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a8105d80 msgr2=0x7f33a819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a6d9d640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a8105d80 0x7f33a819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a6d9d640 1 -- 172.21.15.129:0/3222557408 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a81a2a60 con 0x7f33a81086b0 2024-04-26T21:24:34.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a8105d80 0x7f33a819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:34.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33a6d9d640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a81086b0 0x7f33a81a23f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f33a000eb10 tx=0x7f33a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:34.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.560+0000 7f33977fe640 1 -- 172.21.15.129:0/3222557408 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004b20 con 0x7f33a81086b0 2024-04-26T21:24:34.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.561+0000 7f33977fe640 1 -- 172.21.15.129:0/3222557408 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a0004cc0 con 0x7f33a81086b0 2024-04-26T21:24:34.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.561+0000 7f33acd14640 1 -- 172.21.15.129:0/3222557408 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a81a2d50 con 0x7f33a81086b0 2024-04-26T21:24:34.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.561+0000 7f33977fe640 1 -- 172.21.15.129:0/3222557408 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0010670 con 0x7f33a81086b0 2024-04-26T21:24:34.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.561+0000 7f33acd14640 1 -- 172.21.15.129:0/3222557408 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a81a31f0 con 0x7f33a81086b0 2024-04-26T21:24:34.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33977fe640 1 -- 172.21.15.129:0/3222557408 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f33a0005020 con 0x7f33a81086b0 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33afa2c740 1 Processor -- start 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33afa2c740 1 -- start start 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 0x5602aff36300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff370b0 0x5602aff37530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff37a70 0x5602aff3e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33afa2c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5602afe9ad80 con 0x5602aff370b0 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33afa2c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5602afe548b0 con 0x5602aff37a70 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33afa2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5602afe96d40 con 0x5602aff35ee0 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33a659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff370b0 0x5602aff37530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33a5d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 0x5602aff36300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33a659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff370b0 0x5602aff37530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50570/0 (socket says 172.21.15.129:50570) 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33a659c640 1 -- 172.21.15.129:0/416546677 learned_addr learned my addr 172.21.15.129:0/416546677 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.564+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/416546677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff37a70 0x5602aff3e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f33977fe640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33900a3c40 0x7f33900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f33a659c640 1 -- 172.21.15.129:0/416546677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 msgr2=0x5602aff36300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f33a659c640 1 --2- 172.21.15.129:0/416546677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 0x5602aff36300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f33a659c640 1 -- 172.21.15.129:0/416546677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff37a70 msgr2=0x5602aff3e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f33a659c640 1 --2- 172.21.15.129:0/416546677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff37a70 0x5602aff3e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f33a659c640 1 -- 172.21.15.129:0/416546677 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602afe25a40 con 0x5602aff370b0 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f33977fe640 1 -- 172.21.15.129:0/3222557408 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f33a00d0b50 con 0x7f33a81086b0 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f33a6d9d640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33900a3c40 0x7f33900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f33a659c640 1 --2- 172.21.15.129:0/416546677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff370b0 0x5602aff37530 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f339c009900 tx=0x7f339c005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d21e76a2552db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f3394ff9640 1 -- 172.21.15.129:0/416546677 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c035030 con 0x5602aff370b0 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.565+0000 7f3394ff9640 1 -- 172.21.15.129:0/416546677 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339c040070 con 0x5602aff370b0 2024-04-26T21:24:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.566+0000 7f3394ff9640 1 -- 172.21.15.129:0/416546677 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c003850 con 0x5602aff370b0 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33afa2c740 1 -- 172.21.15.129:0/416546677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff370b0 msgr2=0x5602aff37530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33afa2c740 1 --2- 172.21.15.129:0/416546677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff370b0 0x5602aff37530 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f339c009900 tx=0x7f339c005df0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33a6d9d640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33900a3c40 0x7f33900a6100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f33a00cea70 tx=0x7f33a000cee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33afa2c740 1 -- 172.21.15.129:0/416546677 shutdown_connections 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33afa2c740 1 --2- 172.21.15.129:0/416546677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff37a70 0x5602aff3e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33afa2c740 1 --2- 172.21.15.129:0/416546677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff370b0 0x5602aff37530 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33afa2c740 1 --2- 172.21.15.129:0/416546677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 0x5602aff36300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33afa2c740 1 -- 172.21.15.129:0/416546677 >> 172.21.15.129:0/416546677 conn(0x5602aff313d0 msgr2=0x5602aff317d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33afa2c740 1 -- 172.21.15.129:0/416546677 shutdown_connections 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.567+0000 7f33afa2c740 1 -- 172.21.15.129:0/416546677 wait complete. 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.568+0000 7f33afa2c740 1 -- start start 2024-04-26T21:24:34.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.568+0000 7f33afa2c740 1 Processor -- start 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.569+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 0x5602aff48b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.569+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.569+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.569+0000 7f33afa2c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5602aff3ea20 con 0x5602aff35ee0 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.569+0000 7f33afa2c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5602afe21d10 con 0x5602aff370b0 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.569+0000 7f33afa2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5602aff3eba0 con 0x5602aff37a70 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.569+0000 7f33afa2c740 10 client.0 _ll_register_callbacks cb 0x5602afe4e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a5d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a5d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34330/0 (socket says 172.21.15.129:34330) 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a5d9b640 1 -- 172.21.15.129:0/3377228984 learned_addr learned my addr 172.21.15.129:0/3377228984 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a659c640 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a659c640 1 -- 172.21.15.129:0/3377228984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 msgr2=0x5602aff500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a659c640 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a659c640 1 -- 172.21.15.129:0/3377228984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 msgr2=0x5602aff48b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a659c640 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 0x5602aff48b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a659c640 1 -- 172.21.15.129:0/3377228984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602afc767d0 con 0x5602aff370b0 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.570+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.571+0000 7f33a659c640 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff490b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f339c000c00 tx=0x7f339c009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8608a3ad2a7426f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.571+0000 7f338dffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.571+0000 7f338dffb640 1 -- 172.21.15.129:0/3377228984 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c035050 con 0x5602aff370b0 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.571+0000 7f33afa2c740 1 -- 172.21.15.129:0/3377228984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339c040070 con 0x5602aff370b0 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.571+0000 7f338dffb640 1 -- 172.21.15.129:0/3377228984 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f339c0035f0 con 0x5602aff370b0 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.571+0000 7f338dffb640 1 -- 172.21.15.129:0/3377228984 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f339c0037f0 con 0x5602aff370b0 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.571+0000 7f33afa2c740 10 client.25520 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.571+0000 7f33afa2c740 1 -- 172.21.15.129:0/3377228984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5602afbac3f0 con 0x5602aff370b0 2024-04-26T21:24:34.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.572+0000 7f338dffb640 1 -- 172.21.15.129:0/3377228984 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339c040070 con 0x5602aff370b0 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.572+0000 7f338dffb640 1 -- 172.21.15.129:0/3377228984 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f339c045880 con 0x5602aff370b0 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.572+0000 7f33afa2c740 10 client.25520 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.572+0000 7f33afa2c740 10 client.25520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.572+0000 7f33afa2c740 20 client.25520 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.572+0000 7f33afa2c740 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5602aff65260 0x5602aff67650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.572+0000 7f33afa2c740 4 client.25520 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.572+0000 7f33afa2c740 1 -- 172.21.15.129:0/3377228984 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602afea8990 con 0x5602aff65260 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.574+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5602aff65260 0x5602aff67650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.578+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5602aff65260 0x5602aff67650 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2662b26a19d47b50 server_cookie=7bded483552a7fd1 in_seq=0 out_seq=0 2024-04-26T21:24:34.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.578+0000 7f338dffb640 10 client.25520 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:34.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f338dffb640 1 -- 172.21.15.129:0/3377228984 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602afea8990 con 0x5602aff65260 2024-04-26T21:24:34.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f338dffb640 10 client.25520 handle_command_reply: tid=0 2024-04-26T21:24:34.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 client.25520 shutdown 2024-04-26T21:24:34.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 -- 172.21.15.129:0/3377228984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 msgr2=0x5602aff490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff490b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f339c000c00 tx=0x7f339c009660 comp rx=0 tx=0).stop 2024-04-26T21:24:34.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 -- 172.21.15.129:0/3377228984 shutdown_connections 2024-04-26T21:24:34.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5602aff65260 0x5602aff67650 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff490b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f338dffb640 0 client.25520 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:34.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 --2- 172.21.15.129:0/3377228984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 0x5602aff48b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 -- 172.21.15.129:0/3377228984 >> 172.21.15.129:0/3377228984 conn(0x5602aff313d0 msgr2=0x5602aff30550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:34.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 -- 172.21.15.129:0/3377228984 shutdown_connections 2024-04-26T21:24:34.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 1 -- 172.21.15.129:0/3377228984 wait complete. 2024-04-26T21:24:34.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.587+0000 7f33afa2c740 20 client.25520 trim_cache size 0 max 16384 2024-04-26T21:24:34.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.620+0000 7f33afa2c740 1 Processor -- start 2024-04-26T21:24:34.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.620+0000 7f33afa2c740 1 -- start start 2024-04-26T21:24:34.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.620+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 0x5602aff3ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.620+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.620+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff3b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.620+0000 7f33afa2c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33a806eb00 con 0x5602aff35ee0 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.620+0000 7f33afa2c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8071360 con 0x5602aff370b0 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.620+0000 7f33afa2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f33a806f150 con 0x5602aff37a70 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 0x5602aff3ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 0x5602aff3ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50596/0 (socket says 172.21.15.129:50596) 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a5d9b640 1 -- 172.21.15.129:0/2202612763 learned_addr learned my addr 172.21.15.129:0/2202612763 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff3b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a5d9b640 1 -- 172.21.15.129:0/2202612763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 msgr2=0x5602aff3b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff3b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a5d9b640 1 -- 172.21.15.129:0/2202612763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 msgr2=0x5602aff500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:34.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a659c640 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.621+0000 7f33a5d9b640 1 -- 172.21.15.129:0/2202612763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602afb6c0f0 con 0x5602aff35ee0 2024-04-26T21:24:34.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.622+0000 7f33a659c640 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:24:34.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.622+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 0x5602aff3ab90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5602aff67df0 tx=0x7f33980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374068e5ee4d43d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:34.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.623+0000 7f338ffff640 1 -- 172.21.15.129:0/2202612763 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398023550 con 0x5602aff35ee0 2024-04-26T21:24:34.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.623+0000 7f338ffff640 1 -- 172.21.15.129:0/2202612763 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33980236f0 con 0x5602aff35ee0 2024-04-26T21:24:34.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.623+0000 7f338ffff640 1 -- 172.21.15.129:0/2202612763 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339801ed90 con 0x5602aff35ee0 2024-04-26T21:24:34.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.624+0000 7f33afa2c740 1 -- 172.21.15.129:0/2202612763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 msgr2=0x5602aff3ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.624+0000 7f33afa2c740 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 0x5602aff3ab90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5602aff67df0 tx=0x7f33980046c0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.624+0000 7f33afa2c740 1 -- 172.21.15.129:0/2202612763 shutdown_connections 2024-04-26T21:24:34.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.624+0000 7f33afa2c740 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.624+0000 7f33afa2c740 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff37a70 0x5602aff3b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.624+0000 7f33afa2c740 1 --2- 172.21.15.129:0/2202612763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff35ee0 0x5602aff3ab90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.624+0000 7f33afa2c740 1 -- 172.21.15.129:0/2202612763 >> 172.21.15.129:0/2202612763 conn(0x5602aff313d0 msgr2=0x5602aff2c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:34.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.624+0000 7f33afa2c740 1 -- 172.21.15.129:0/2202612763 shutdown_connections 2024-04-26T21:24:34.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.624+0000 7f33afa2c740 1 -- 172.21.15.129:0/2202612763 wait complete. 2024-04-26T21:24:34.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.625+0000 7f33afa2c740 1 -- start start 2024-04-26T21:24:34.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.625+0000 7f33afa2c740 1 Processor -- start 2024-04-26T21:24:34.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.625+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 0x5602aff4d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.625+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff4d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.625+0000 7f33afa2c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff37a70 0x5602affc9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.625+0000 7f33afa2c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33a806f820 con 0x5602aff37a70 2024-04-26T21:24:34.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.625+0000 7f33afa2c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8070c90 con 0x5602aff370b0 2024-04-26T21:24:34.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.625+0000 7f33afa2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f33a806fef0 con 0x5602aff35ee0 2024-04-26T21:24:34.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.625+0000 7f33afa2c740 10 client.0 _ll_register_callbacks cb 0x7f339c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff4d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff4d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:56974/0 (socket says 172.21.15.129:56974) 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a659c640 1 -- 172.21.15.129:0/409506657 learned_addr learned my addr 172.21.15.129:0/409506657 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 0x5602aff4d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff37a70 0x5602affc9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a659c640 1 -- 172.21.15.129:0/409506657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 msgr2=0x5602aff4d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a659c640 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 0x5602aff4d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a659c640 1 -- 172.21.15.129:0/409506657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff37a70 msgr2=0x5602affc9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a659c640 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff37a70 0x5602affc9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.626+0000 7f33a659c640 1 -- 172.21.15.129:0/409506657 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602afe5b4b0 con 0x5602aff370b0 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.627+0000 7f33a659c640 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff4d560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f339c045d20 tx=0x7f339c0095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c82eddd3d91796d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:34.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.627+0000 7f338dffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:24:34.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.627+0000 7f338dffb640 1 -- 172.21.15.129:0/409506657 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c05f020 con 0x5602aff370b0 2024-04-26T21:24:34.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.627+0000 7f338dffb640 1 -- 172.21.15.129:0/409506657 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f339c045670 con 0x5602aff370b0 2024-04-26T21:24:34.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.627+0000 7f33afa2c740 1 -- 172.21.15.129:0/409506657 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33980236f0 con 0x5602aff370b0 2024-04-26T21:24:34.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.627+0000 7f338dffb640 1 -- 172.21.15.129:0/409506657 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f339c06a050 con 0x5602aff370b0 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f33afa2c740 10 client.25523 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f33afa2c740 1 -- 172.21.15.129:0/409506657 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5602affcad80 con 0x5602aff370b0 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f338dffb640 1 -- 172.21.15.129:0/409506657 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33980236f0 con 0x5602aff370b0 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f338dffb640 1 -- 172.21.15.129:0/409506657 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f339c03c650 con 0x5602aff370b0 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f33afa2c740 10 client.25523 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f33afa2c740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f33afa2c740 20 client.25523 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f33afa2c740 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5602affcb700 0x5602affcdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f33afa2c740 4 client.25523 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:24:34.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.628+0000 7f33afa2c740 1 -- 172.21.15.129:0/409506657 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5602afea2800 con 0x5602affcb700 2024-04-26T21:24:34.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.631+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5602affcb700 0x5602affcdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.632+0000 7f338dffb640 10 client.25523 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:34.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.632+0000 7f33a5d9b640 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5602affcb700 0x5602affcdb20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2de4bd5f516880 server_cookie=c00f0cce37ea730a in_seq=0 out_seq=0 2024-04-26T21:24:34.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.637+0000 7f338dffb640 1 -- 172.21.15.129:0/409506657 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5602afea2800 con 0x5602affcb700 2024-04-26T21:24:34.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f338dffb640 10 client.25523 handle_command_reply: tid=0 2024-04-26T21:24:34.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 client.25523 shutdown 2024-04-26T21:24:34.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 -- 172.21.15.129:0/409506657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 msgr2=0x5602aff4d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff4d560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f339c045d20 tx=0x7f339c0095a0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 -- 172.21.15.129:0/409506657 shutdown_connections 2024-04-26T21:24:34.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5602affcb700 0x5602affcdb20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602aff370b0 0x5602aff4d560 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f339c045d20 tx=0x7f339c0095a0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602aff37a70 0x5602affc9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 --2- 172.21.15.129:0/409506657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5602aff35ee0 0x5602aff4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 -- 172.21.15.129:0/409506657 >> 172.21.15.129:0/409506657 conn(0x5602aff313d0 msgr2=0x5602aff48b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:34.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f338dffb640 0 client.25523 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:34.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.638+0000 7f33afa2c740 1 -- 172.21.15.129:0/409506657 shutdown_connections 2024-04-26T21:24:34.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.640+0000 7f33afa2c740 1 -- 172.21.15.129:0/409506657 wait complete. 2024-04-26T21:24:34.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.640+0000 7f33afa2c740 20 client.25523 trim_cache size 0 max 16384 2024-04-26T21:24:34.642 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:24:34.642 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:24:34.642 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:24:34.642 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:24:34.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.642+0000 7f33acd14640 1 -- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33900a3c40 msgr2=0x7f33900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.642+0000 7f33acd14640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33900a3c40 0x7f33900a6100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f33a00cea70 tx=0x7f33a000cee0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.642+0000 7f33acd14640 1 -- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a81086b0 msgr2=0x7f33a81a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.642+0000 7f33acd14640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a81086b0 0x7f33a81a23f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f33a000eb10 tx=0x7f33a000c2d0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.642+0000 7f33acd14640 1 -- 172.21.15.129:0/3222557408 shutdown_connections 2024-04-26T21:24:34.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.642+0000 7f33acd14640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f33900a3c40 0x7f33900a6100 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.643+0000 7f33acd14640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a8105d80 0x7f33a819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.643+0000 7f33acd14640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a81086b0 0x7f33a81a23f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.643+0000 7f33acd14640 1 --2- 172.21.15.129:0/3222557408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a8103450 0x7f33a819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.643+0000 7f33acd14640 1 -- 172.21.15.129:0/3222557408 >> 172.21.15.129:0/3222557408 conn(0x7f33a80fcde0 msgr2=0x7f33a8102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:34.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.644+0000 7f33acd14640 1 -- 172.21.15.129:0/3222557408 shutdown_connections 2024-04-26T21:24:34.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.645+0000 7f33acd14640 1 -- 172.21.15.129:0/3222557408 wait complete. 2024-04-26T21:24:34.669 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:24:34.669 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:24:34.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 -- 172.21.15.129:0/1373502695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc101870 msgr2=0x7f8fbc101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/1373502695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc101870 0x7f8fbc101c90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009900 tx=0x7f8fb002a410 comp rx=0 tx=0).stop 2024-04-26T21:24:34.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 -- 172.21.15.129:0/1373502695 shutdown_connections 2024-04-26T21:24:34.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/1373502695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fbc103420 0x7f8fbc109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/1373502695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fbc102a60 0x7f8fbc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/1373502695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc101870 0x7f8fbc101c90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 -- 172.21.15.129:0/1373502695 >> 172.21.15.129:0/1373502695 conn(0x7f8fbc0fcde0 msgr2=0x7f8fbc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:34.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 -- 172.21.15.129:0/1373502695 shutdown_connections 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 -- 172.21.15.129:0/1373502695 wait complete. 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 Processor -- start 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.774+0000 7f8fc2b86640 1 -- start start 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc2b86640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fbc101870 0x7f8fbc19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc2b86640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fbc102a60 0x7f8fbc19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc2b86640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc103420 0x7f8fbc1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc2b86640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc10d390 con 0x7f8fbc103420 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc2b86640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc10d210 con 0x7f8fbc102a60 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc2b86640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc10d510 con 0x7f8fbc101870 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc1123640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc103420 0x7f8fbc1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc1123640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc103420 0x7f8fbc1a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50626/0 (socket says 172.21.15.129:50626) 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc1123640 1 -- 172.21.15.129:0/3010365537 learned_addr learned my addr 172.21.15.129:0/3010365537 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:34.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fbbfff640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fbc102a60 0x7f8fbc19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc0922640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fbc101870 0x7f8fbc19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc1123640 1 -- 172.21.15.129:0/3010365537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fbc101870 msgr2=0x7f8fbc19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc1123640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fbc101870 0x7f8fbc19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc1123640 1 -- 172.21.15.129:0/3010365537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fbc102a60 msgr2=0x7f8fbc19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc1123640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fbc102a60 0x7f8fbc19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc1123640 1 -- 172.21.15.129:0/3010365537 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc1a2a20 con 0x7f8fbc103420 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc1123640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc103420 0x7f8fbc1a23b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400efd0 tx=0x7f8fb400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fb9ffb640 1 -- 172.21.15.129:0/3010365537 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4003b70 con 0x7f8fbc103420 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fb9ffb640 1 -- 172.21.15.129:0/3010365537 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb4003d10 con 0x7f8fbc103420 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fb9ffb640 1 -- 172.21.15.129:0/3010365537 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40108a0 con 0x7f8fbc103420 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.775+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc1a2d10 con 0x7f8fbc103420 2024-04-26T21:24:34.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.776+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc1a31b0 con 0x7f8fbc103420 2024-04-26T21:24:34.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.777+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc1a3440 con 0x7f8fbc103420 2024-04-26T21:24:34.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.778+0000 7f8fb9ffb640 1 -- 172.21.15.129:0/3010365537 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8fb4004020 con 0x7f8fbc103420 2024-04-26T21:24:34.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.778+0000 7f8fb9ffb640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8fac0a3c40 0x7f8fac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:34.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.778+0000 7f8fc0922640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8fac0a3c40 0x7f8fac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:34.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.779+0000 7f8fb9ffb640 1 -- 172.21.15.129:0/3010365537 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f8fb40d0980 con 0x7f8fbc103420 2024-04-26T21:24:34.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.779+0000 7f8fc0922640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8fac0a3c40 0x7f8fac0a6100 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009230 tx=0x7f8fb00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:34.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:34.783+0000 7f8fb9ffb640 1 -- 172.21.15.129:0/3010365537 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8fb409d1f0 con 0x7f8fbc103420 2024-04-26T21:24:34.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:34 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:34.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:34 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:34.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:34 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:35.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.069+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fbc1a3860 con 0x7f8fbc103420 2024-04-26T21:24:35.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.070+0000 7f8fb9ffb640 1 -- 172.21.15.129:0/3010365537 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f8fb409caf0 con 0x7f8fbc103420 2024-04-26T21:24:35.071 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:24:35.072 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:24:35.072 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.072+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8fac0a3c40 msgr2=0x7f8fac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.072+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8fac0a3c40 0x7f8fac0a6100 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009230 tx=0x7f8fb00314a0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.072+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc103420 msgr2=0x7f8fbc1a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.072+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc103420 0x7f8fbc1a23b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400efd0 tx=0x7f8fb400c6a0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.073+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 shutdown_connections 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.073+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8fac0a3c40 0x7f8fac0a6100 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.073+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fbc102a60 0x7f8fbc19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.073+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fbc103420 0x7f8fbc1a23b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.073+0000 7f8fc2b86640 1 --2- 172.21.15.129:0/3010365537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fbc101870 0x7f8fbc19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.073+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 >> 172.21.15.129:0/3010365537 conn(0x7f8fbc0fcde0 msgr2=0x7f8fbc0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:35.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.073+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 shutdown_connections 2024-04-26T21:24:35.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.073+0000 7f8fc2b86640 1 -- 172.21.15.129:0/3010365537 wait complete. 2024-04-26T21:24:35.086 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:24:35.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 -- 172.21.15.129:0/1247116541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f81033f0 msgr2=0x7f71f81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 --2- 172.21.15.129:0/1247116541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f81033f0 0x7f71f81057e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009900 tx=0x7f71e802a410 comp rx=0 tx=0).stop 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 -- 172.21.15.129:0/1247116541 shutdown_connections 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 --2- 172.21.15.129:0/1247116541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f8108650 0x7f71f810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 --2- 172.21.15.129:0/1247116541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f8105d20 0x7f71f8108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 --2- 172.21.15.129:0/1247116541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f81033f0 0x7f71f81057e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 -- 172.21.15.129:0/1247116541 >> 172.21.15.129:0/1247116541 conn(0x7f71f80fcde0 msgr2=0x7f71f80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 -- 172.21.15.129:0/1247116541 shutdown_connections 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 -- 172.21.15.129:0/1247116541 wait complete. 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.197+0000 7f71fc946640 1 Processor -- start 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71fc946640 1 -- start start 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71fc946640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f81033f0 0x7f71f8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71fc946640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f8105d20 0x7f71f8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71fc946640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f8108650 0x7f71f81a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71fc946640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71f810e4a0 con 0x7f71f8108650 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71fc946640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f71f810e320 con 0x7f71f81033f0 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71fc946640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71f810e620 con 0x7f71f8105d20 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f8108650 0x7f71f81a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f81033f0 0x7f71f8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f8108650 0x7f71f81a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50642/0 (socket says 172.21.15.129:50642) 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f81033f0 0x7f71f8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:56998/0 (socket says 172.21.15.129:56998) 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f6d9d640 1 -- 172.21.15.129:0/570907935 learned_addr learned my addr 172.21.15.129:0/570907935 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f8105d20 0x7f71f8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f6d9d640 1 -- 172.21.15.129:0/570907935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f8105d20 msgr2=0x7f71f8199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f6d9d640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f8105d20 0x7f71f8199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f6d9d640 1 -- 172.21.15.129:0/570907935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f81033f0 msgr2=0x7f71f8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f6d9d640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f81033f0 0x7f71f8198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f6d9d640 1 -- 172.21.15.129:0/570907935 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f81a0860 con 0x7f71f8108650 2024-04-26T21:24:35.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f8105d20 0x7f71f8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:35.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f659c640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f81033f0 0x7f71f8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:35.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71f6d9d640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f8108650 0x7f71f81a01c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f71f000eb10 tx=0x7f71f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:35.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.198+0000 7f71e77fe640 1 -- 172.21.15.129:0/570907935 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004b20 con 0x7f71f8108650 2024-04-26T21:24:35.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.199+0000 7f71fc946640 1 -- 172.21.15.129:0/570907935 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f81a0b50 con 0x7f71f8108650 2024-04-26T21:24:35.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.199+0000 7f71e77fe640 1 -- 172.21.15.129:0/570907935 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f0004cc0 con 0x7f71f8108650 2024-04-26T21:24:35.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.199+0000 7f71e77fe640 1 -- 172.21.15.129:0/570907935 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0010670 con 0x7f71f8108650 2024-04-26T21:24:35.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.199+0000 7f71fc946640 1 -- 172.21.15.129:0/570907935 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f81a10a0 con 0x7f71f8108650 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.201+0000 7f71e77fe640 1 -- 172.21.15.129:0/570907935 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f71f0005020 con 0x7f71f8108650 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.201+0000 7f71ff665740 1 Processor -- start 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.201+0000 7f71ff665740 1 -- start start 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71ff665740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 0x557de61ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71ff665740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71ff665740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ae7c0 0x557de61b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71ff665740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557de610cd80 con 0x557de61ade00 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71ff665740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557de60c6a20 con 0x557de61acc30 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71ff665740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557de6108d40 con 0x557de61ae7c0 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71e77fe640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f71e00a3c40 0x7f71e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f5d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 0x557de61ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50646/0 (socket says 172.21.15.129:50646) 2024-04-26T21:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f659c640 1 -- 172.21.15.129:0/2889781600 learned_addr learned my addr 172.21.15.129:0/2889781600 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/2889781600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ae7c0 0x557de61b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f6d9d640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f71e00a3c40 0x7f71e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71e77fe640 1 -- 172.21.15.129:0/570907935 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f71f00d0be0 con 0x7f71f8108650 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f5d9b640 1 -- 172.21.15.129:0/2889781600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ae7c0 msgr2=0x557de61b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/2889781600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ae7c0 0x557de61b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f5d9b640 1 -- 172.21.15.129:0/2889781600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 msgr2=0x557de61ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/2889781600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f5d9b640 1 -- 172.21.15.129:0/2889781600 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de60d1c30 con 0x557de61acc30 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f659c640 1 --2- 172.21.15.129:0/2889781600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/2889781600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 0x557de61ad050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f71f806a620 tx=0x7f71ec005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e22c4723893774e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71f6d9d640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f71e00a3c40 0x7f71e00a6100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f71f00ceb00 tx=0x7f71f000cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71e4ff9640 1 -- 172.21.15.129:0/2889781600 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec011050 con 0x557de61acc30 2024-04-26T21:24:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71e4ff9640 1 -- 172.21.15.129:0/2889781600 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ec00fdf0 con 0x557de61acc30 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.202+0000 7f71e4ff9640 1 -- 172.21.15.129:0/2889781600 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0055f0 con 0x557de61acc30 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.203+0000 7f71ff665740 1 -- 172.21.15.129:0/2889781600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 msgr2=0x557de61ad050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.203+0000 7f71ff665740 1 --2- 172.21.15.129:0/2889781600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 0x557de61ad050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f71f806a620 tx=0x7f71ec005850 comp rx=0 tx=0).stop 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.203+0000 7f71ff665740 1 -- 172.21.15.129:0/2889781600 shutdown_connections 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.203+0000 7f71ff665740 1 --2- 172.21.15.129:0/2889781600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ae7c0 0x557de61b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.203+0000 7f71ff665740 1 --2- 172.21.15.129:0/2889781600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.203+0000 7f71ff665740 1 --2- 172.21.15.129:0/2889781600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 0x557de61ad050 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.203+0000 7f71ff665740 1 -- 172.21.15.129:0/2889781600 >> 172.21.15.129:0/2889781600 conn(0x557de60e9910 msgr2=0x557de61a4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.203+0000 7f71ff665740 1 -- 172.21.15.129:0/2889781600 shutdown_connections 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.203+0000 7f71ff665740 1 -- 172.21.15.129:0/2889781600 wait complete. 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ff665740 1 -- start start 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ff665740 1 Processor -- start 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ff665740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 0x557de61b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ff665740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ade00 0x557de61b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ff665740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61be460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ff665740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557de61b57d0 con 0x557de61ae7c0 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ff665740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557de6093dc0 con 0x557de61ade00 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ff665740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557de61b5950 con 0x557de61acc30 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ff665740 10 client.0 _ll_register_callbacks cb 0x557de60af600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ade00 0x557de61b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61be460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ade00 0x557de61b7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:57012/0 (socket says 172.21.15.129:57012) 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f659c640 1 -- 172.21.15.129:0/3558507183 learned_addr learned my addr 172.21.15.129:0/3558507183 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 0x557de61b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f659c640 1 -- 172.21.15.129:0/3558507183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 msgr2=0x557de61b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f659c640 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 0x557de61b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f659c640 1 -- 172.21.15.129:0/3558507183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 msgr2=0x557de61be460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f659c640 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61be460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f659c640 1 -- 172.21.15.129:0/3558507183 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de5e1e3f0 con 0x557de61ade00 2024-04-26T21:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61be460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71f659c640 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ade00 0x557de61b7420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71e8033000 tx=0x7f71e80317d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=940d5a5b211b12b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.204+0000 7f71ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ddffb640 1 -- 172.21.15.129:0/3558507183 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8035040 con 0x557de61ade00 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ddffb640 1 -- 172.21.15.129:0/3558507183 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f71e8004240 con 0x557de61ade00 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ff665740 1 -- 172.21.15.129:0/3558507183 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ec00fdf0 con 0x557de61ade00 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ddffb640 1 -- 172.21.15.129:0/3558507183 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f71e8004440 con 0x557de61ade00 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ff665740 10 client.25541 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ff665740 1 -- 172.21.15.129:0/3558507183 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557de5ee87d0 con 0x557de61ade00 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ddffb640 1 -- 172.21.15.129:0/3558507183 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ec00fdf0 con 0x557de61ade00 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ddffb640 1 -- 172.21.15.129:0/3558507183 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f71e8004c20 con 0x557de61ade00 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ff665740 10 client.25541 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ff665740 10 client.25541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ff665740 20 client.25541 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:35.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ff665740 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x557de61c2dd0 0x557de61c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ff665740 4 client.25541 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:24:35.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71ff665740 1 -- 172.21.15.129:0/3558507183 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557de611a990 con 0x557de61c2dd0 2024-04-26T21:24:35.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.205+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x557de61c2dd0 0x557de61c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.206+0000 7f71ddffb640 10 client.25541 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:35.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.206+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x557de61c2dd0 0x557de61c51c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1cd78ccea394a5d server_cookie=f084f00b0d46cd07 in_seq=0 out_seq=0 2024-04-26T21:24:35.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ddffb640 1 -- 172.21.15.129:0/3558507183 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557de611a990 con 0x557de61c2dd0 2024-04-26T21:24:35.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ddffb640 10 client.25541 handle_command_reply: tid=0 2024-04-26T21:24:35.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 client.25541 shutdown 2024-04-26T21:24:35.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 -- 172.21.15.129:0/3558507183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ade00 msgr2=0x557de61b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ade00 0x557de61b7420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71e8033000 tx=0x7f71e80317d0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 -- 172.21.15.129:0/3558507183 shutdown_connections 2024-04-26T21:24:35.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x557de61c2dd0 0x557de61c51c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ade00 0x557de61b7420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ddffb640 0 client.25541 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:35.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61be460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 --2- 172.21.15.129:0/3558507183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 0x557de61b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 -- 172.21.15.129:0/3558507183 >> 172.21.15.129:0/3558507183 conn(0x557de60e9910 msgr2=0x557de61a3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:35.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 -- 172.21.15.129:0/3558507183 shutdown_connections 2024-04-26T21:24:35.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 1 -- 172.21.15.129:0/3558507183 wait complete. 2024-04-26T21:24:35.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.209+0000 7f71ff665740 20 client.25541 trim_cache size 0 max 16384 2024-04-26T21:24:35.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.222+0000 7f71ff665740 1 Processor -- start 2024-04-26T21:24:35.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.222+0000 7f71ff665740 1 -- start start 2024-04-26T21:24:35.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.222+0000 7f71ff665740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 0x557de61a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.222+0000 7f71ff665740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61be420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.222+0000 7f71ff665740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ae7c0 0x557de61a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.222+0000 7f71ff665740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81a5ba0 con 0x557de61ade00 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.222+0000 7f71ff665740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81a8420 con 0x557de61ae7c0 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.222+0000 7f71ff665740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81a6210 con 0x557de61acc30 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61be420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61be420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50662/0 (socket says 172.21.15.129:50662) 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f659c640 1 -- 172.21.15.129:0/547217092 learned_addr learned my addr 172.21.15.129:0/547217092 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ae7c0 0x557de61a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 0x557de61a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f659c640 1 -- 172.21.15.129:0/547217092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 msgr2=0x557de61a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f659c640 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 0x557de61a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f659c640 1 -- 172.21.15.129:0/547217092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ae7c0 msgr2=0x557de61a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:35.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f659c640 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ae7c0 0x557de61a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f659c640 1 -- 172.21.15.129:0/547217092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de61ad880 con 0x557de61ade00 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 0x557de61a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71f659c640 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61be420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009850 tx=0x7f71e8005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de04ec403254c05f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71dffff640 1 -- 172.21.15.129:0/547217092 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8035030 con 0x557de61ade00 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71dffff640 1 -- 172.21.15.129:0/547217092 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e8003a30 con 0x557de61ade00 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.223+0000 7f71dffff640 1 -- 172.21.15.129:0/547217092 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80046c0 con 0x557de61ade00 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 -- 172.21.15.129:0/547217092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 msgr2=0x557de61be420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61be420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009850 tx=0x7f71e8005e00 comp rx=0 tx=0).stop 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 -- 172.21.15.129:0/547217092 shutdown_connections 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61ae7c0 0x557de61a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ade00 0x557de61be420 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 --2- 172.21.15.129:0/547217092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61acc30 0x557de61a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 -- 172.21.15.129:0/547217092 >> 172.21.15.129:0/547217092 conn(0x557de60e9910 msgr2=0x557de61b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 -- 172.21.15.129:0/547217092 shutdown_connections 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 -- 172.21.15.129:0/547217092 wait complete. 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 -- start start 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 Processor -- start 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 0x557de61bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ade00 0x557de61bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61bce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81a68e0 con 0x557de61ae7c0 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81a7d50 con 0x557de61acc30 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81a6fb0 con 0x557de61ade00 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.224+0000 7f71ff665740 10 client.0 _ll_register_callbacks cb 0x7f71e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 0x557de61bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61bce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61bce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50668/0 (socket says 172.21.15.129:50668) 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 -- 172.21.15.129:0/1717193630 learned_addr learned my addr 172.21.15.129:0/1717193630 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f659c640 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ade00 0x557de61bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 -- 172.21.15.129:0/1717193630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ade00 msgr2=0x557de61bbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ade00 0x557de61bbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 -- 172.21.15.129:0/1717193630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 msgr2=0x557de61bb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 0x557de61bb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 -- 172.21.15.129:0/1717193630 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de61bd3f0 con 0x557de61ae7c0 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71f5d9b640 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61bce00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f71ec013010 tx=0x7f71ec0118b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac7573d3f4513fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71ddffb640 1 -- 172.21.15.129:0/1717193630 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec000e60 con 0x557de61ae7c0 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71ddffb640 1 -- 172.21.15.129:0/1717193630 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f71ec029040 con 0x557de61ae7c0 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71ff665740 1 -- 172.21.15.129:0/1717193630 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e8003a30 con 0x557de61ae7c0 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71ddffb640 1 -- 172.21.15.129:0/1717193630 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f71ec005340 con 0x557de61ae7c0 2024-04-26T21:24:35.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71ff665740 10 client.15954 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71ff665740 1 -- 172.21.15.129:0/1717193630 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557de61be6f0 con 0x557de61ae7c0 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.225+0000 7f71ddffb640 1 -- 172.21.15.129:0/1717193630 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e8003a30 con 0x557de61ae7c0 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71ddffb640 1 -- 172.21.15.129:0/1717193630 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f71ec0038c0 con 0x557de61ae7c0 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71ff665740 10 client.15954 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71ff665740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71ff665740 20 client.15954 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71ff665740 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x557de61c2dd0 0x557de622b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71ff665740 4 client.15954 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71ff665740 1 -- 172.21.15.129:0/1717193630 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557de6114800 con 0x557de61c2dd0 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71f659c640 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x557de61c2dd0 0x557de622b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71f659c640 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x557de61c2dd0 0x557de622b890 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e5bc0f4ba448d35 server_cookie=b382eade620a8524 in_seq=0 out_seq=0 2024-04-26T21:24:35.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.226+0000 7f71ddffb640 10 client.15954 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ddffb640 1 -- 172.21.15.129:0/1717193630 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557de6114800 con 0x557de61c2dd0 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ddffb640 10 client.15954 handle_command_reply: tid=0 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 client.15954 shutdown 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 -- 172.21.15.129:0/1717193630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 msgr2=0x557de61bce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61bce00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f71ec013010 tx=0x7f71ec0118b0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 -- 172.21.15.129:0/1717193630 shutdown_connections 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x557de61c2dd0 0x557de622b890 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557de61ae7c0 0x557de61bce00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557de61ade00 0x557de61bbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 --2- 172.21.15.129:0/1717193630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de61acc30 0x557de61bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ddffb640 0 client.15954 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:35.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 -- 172.21.15.129:0/1717193630 >> 172.21.15.129:0/1717193630 conn(0x557de60e9910 msgr2=0x557de61b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:35.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 -- 172.21.15.129:0/1717193630 shutdown_connections 2024-04-26T21:24:35.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 1 -- 172.21.15.129:0/1717193630 wait complete. 2024-04-26T21:24:35.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71ff665740 20 client.15954 trim_cache size 0 max 16384 2024-04-26T21:24:35.236 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:24:35.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71fc946640 1 -- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f71e00a3c40 msgr2=0x7f71e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.230+0000 7f71fc946640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f71e00a3c40 0x7f71e00a6100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f71f00ceb00 tx=0x7f71f000cf70 comp rx=0 tx=0).stop 2024-04-26T21:24:35.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 -- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f8108650 msgr2=0x7f71f81a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:35.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f8108650 0x7f71f81a01c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f71f000eb10 tx=0x7f71f000c2d0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 -- 172.21.15.129:0/570907935 shutdown_connections 2024-04-26T21:24:35.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f71e00a3c40 0x7f71e00a6100 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71f8108650 0x7f71f81a01c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f8105d20 0x7f71f8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 --2- 172.21.15.129:0/570907935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f81033f0 0x7f71f8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:35.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 -- 172.21.15.129:0/570907935 >> 172.21.15.129:0/570907935 conn(0x7f71f80fcde0 msgr2=0x7f71f81052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:35.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 -- 172.21.15.129:0/570907935 shutdown_connections 2024-04-26T21:24:35.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:35.231+0000 7f71fc946640 1 -- 172.21.15.129:0/570907935 wait complete. 2024-04-26T21:24:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3010365537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:24:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi129 to 176.2M 2024-04-26T21:24:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: Unable to set osd_memory_target on smithi129 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-26T21:24:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:24:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:35 smithi150 ceph-mon[32290]: pgmap v548: 97 pgs: 97 active+clean; 812 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 495 op/s 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3010365537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi129 to 176.2M 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: Unable to set osd_memory_target on smithi129 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:24:35.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:35 smithi129 ceph-mon[29237]: pgmap v548: 97 pgs: 97 active+clean; 812 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 495 op/s 2024-04-26T21:24:35.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3010365537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:24:35.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:35.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi129 to 176.2M 2024-04-26T21:24:35.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: Unable to set osd_memory_target on smithi129 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-26T21:24:35.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:35.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:35.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:24:35.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:35 smithi171 ceph-mon[33478]: pgmap v548: 97 pgs: 97 active+clean; 812 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 495 op/s 2024-04-26T21:24:36.243 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:24:36.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:24:36.243 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:24:37.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.097+0000 7f018934a640 1 -- 172.21.15.129:0/1192400214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0184109f10 msgr2=0x7f018410a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:37.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.097+0000 7f018934a640 1 --2- 172.21.15.129:0/1192400214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0184109f10 0x7f018410a390 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0178009860 tx=0x7f017802a340 comp rx=0 tx=0).stop 2024-04-26T21:24:37.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.138+0000 7f018934a640 1 -- 172.21.15.129:0/1192400214 shutdown_connections 2024-04-26T21:24:37.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.138+0000 7f018934a640 1 --2- 172.21.15.129:0/1192400214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f018410a8d0 0x7f018410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.138+0000 7f018934a640 1 --2- 172.21.15.129:0/1192400214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0184109f10 0x7f018410a390 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.138+0000 7f018934a640 1 --2- 172.21.15.129:0/1192400214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0184073c10 0x7f0184074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.138+0000 7f018934a640 1 -- 172.21.15.129:0/1192400214 >> 172.21.15.129:0/1192400214 conn(0x7f018406f780 msgr2=0x7f0184071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:37.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.151+0000 7f018934a640 1 -- 172.21.15.129:0/1192400214 shutdown_connections 2024-04-26T21:24:37.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.157+0000 7f018934a640 1 -- 172.21.15.129:0/1192400214 wait complete. 2024-04-26T21:24:37.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.157+0000 7f018934a640 1 Processor -- start 2024-04-26T21:24:37.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.158+0000 7f018934a640 1 -- start start 2024-04-26T21:24:37.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.158+0000 7f018934a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0184073c10 0x7f01841a7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.158+0000 7f018934a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184109f10 0x7f01841a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.158+0000 7f018934a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f018410a8d0 0x7f01841aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.158+0000 7f018934a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f018411d170 con 0x7f0184073c10 2024-04-26T21:24:37.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.158+0000 7f018934a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f018411cff0 con 0x7f0184109f10 2024-04-26T21:24:37.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.158+0000 7f018934a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f018411d2f0 con 0x7f018410a8d0 2024-04-26T21:24:37.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0183fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0184073c10 0x7f01841a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0188b49640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f018410a8d0 0x7f01841aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0188b49640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f018410a8d0 0x7f01841aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34452/0 (socket says 172.21.15.129:34452) 2024-04-26T21:24:37.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0188b49640 1 -- 172.21.15.129:0/3330353594 learned_addr learned my addr 172.21.15.129:0/3330353594 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:37.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0188b49640 1 -- 172.21.15.129:0/3330353594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184109f10 msgr2=0x7f01841a7d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:24:37.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0188b49640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184109f10 0x7f01841a7d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0188b49640 1 -- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0184073c10 msgr2=0x7f01841a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:37.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0188b49640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0184073c10 0x7f01841a7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0188b49640 1 -- 172.21.15.129:0/3330353594 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0184111190 con 0x7f018410a8d0 2024-04-26T21:24:37.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.159+0000 7f0183fff640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0184073c10 0x7f01841a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:37.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.160+0000 7f0188b49640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f018410a8d0 0x7f01841aedc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f017c00eda0 tx=0x7f017c00c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:37.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.216+0000 7f01817fa640 1 -- 172.21.15.129:0/3330353594 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c003b70 con 0x7f018410a8d0 2024-04-26T21:24:37.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.216+0000 7f018934a640 1 -- 172.21.15.129:0/3330353594 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0184111480 con 0x7f018410a8d0 2024-04-26T21:24:37.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.216+0000 7f018934a640 1 -- 172.21.15.129:0/3330353594 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01841119d0 con 0x7f018410a8d0 2024-04-26T21:24:37.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.220+0000 7f018d39b740 1 Processor -- start 2024-04-26T21:24:37.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.222+0000 7f018d39b740 1 -- start start 2024-04-26T21:24:37.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f018d39b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 0x55587473cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f018d39b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x555874741fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f018d39b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 0x555874744ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f018d39b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5558746a1b00 con 0x555874742620 2024-04-26T21:24:37.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f018d39b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55587463d920 con 0x55587473d4b0 2024-04-26T21:24:37.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f018d39b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55587469dac0 con 0x55587473cac0 2024-04-26T21:24:37.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f01817fa640 1 -- 172.21.15.129:0/3330353594 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f017c003d10 con 0x7f018410a8d0 2024-04-26T21:24:37.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f01817fa640 1 -- 172.21.15.129:0/3330353594 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c017870 con 0x7f018410a8d0 2024-04-26T21:24:37.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f0183fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x555874741fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f01837fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 0x555874744ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f01817fa640 1 -- 172.21.15.129:0/3330353594 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f017c017ad0 con 0x7f018410a8d0 2024-04-26T21:24:37.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f0183fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x555874741fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:57034/0 (socket says 172.21.15.129:57034) 2024-04-26T21:24:37.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f0183fff640 1 -- 172.21.15.129:0/2209288015 learned_addr learned my addr 172.21.15.129:0/2209288015 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:37.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.223+0000 7f01837fe640 1 --2- 172.21.15.129:0/2209288015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 0x55587473cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.231+0000 7f0183fff640 1 -- 172.21.15.129:0/2209288015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 msgr2=0x55587473cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:37.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.231+0000 7f0183fff640 1 --2- 172.21.15.129:0/2209288015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 0x55587473cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.231+0000 7f0183fff640 1 -- 172.21.15.129:0/2209288015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 msgr2=0x555874744ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:37.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.231+0000 7f0183fff640 1 --2- 172.21.15.129:0/2209288015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 0x555874744ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.231+0000 7f0183fff640 1 -- 172.21.15.129:0/2209288015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587447b7e0 con 0x55587473d4b0 2024-04-26T21:24:37.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.231+0000 7f0183fff640 1 --2- 172.21.15.129:0/2209288015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x555874741fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0174009e70 tx=0x7f017400e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c10aeef4683aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:37.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.232+0000 7f01817fa640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f016c0a3e70 0x7f016c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.232+0000 7f01837fe640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f016c0a3e70 0x7f016c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.233+0000 7f01837fe640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f016c0a3e70 0x7f016c0a6330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0178005b60 tx=0x7f017802c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:37.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.234+0000 7f01817fa640 1 -- 172.21.15.129:0/3330353594 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f017c0d1700 con 0x7f018410a8d0 2024-04-26T21:24:37.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.239+0000 7f01727fc640 1 -- 172.21.15.129:0/2209288015 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174017070 con 0x55587473d4b0 2024-04-26T21:24:37.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.239+0000 7f01727fc640 1 -- 172.21.15.129:0/2209288015 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0174013920 con 0x55587473d4b0 2024-04-26T21:24:37.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.240+0000 7f018d39b740 1 -- 172.21.15.129:0/2209288015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 msgr2=0x555874741fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:37.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.240+0000 7f018d39b740 1 --2- 172.21.15.129:0/2209288015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x555874741fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0174009e70 tx=0x7f017400e5a0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.240+0000 7f018d39b740 1 -- 172.21.15.129:0/2209288015 shutdown_connections 2024-04-26T21:24:37.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.240+0000 7f018d39b740 1 --2- 172.21.15.129:0/2209288015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x555874741fb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.240+0000 7f018d39b740 1 --2- 172.21.15.129:0/2209288015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 0x555874744ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.240+0000 7f018d39b740 1 --2- 172.21.15.129:0/2209288015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 0x55587473cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.240+0000 7f018d39b740 1 -- 172.21.15.129:0/2209288015 >> 172.21.15.129:0/2209288015 conn(0x555874738130 msgr2=0x555874738530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:37.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.251+0000 7f018d39b740 1 -- 172.21.15.129:0/2209288015 shutdown_connections 2024-04-26T21:24:37.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.251+0000 7f018d39b740 1 -- 172.21.15.129:0/2209288015 wait complete. 2024-04-26T21:24:37.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.252+0000 7f018d39b740 1 -- start start 2024-04-26T21:24:37.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.252+0000 7f018d39b740 1 Processor -- start 2024-04-26T21:24:37.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.252+0000 7f018d39b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x555874746ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.252+0000 7f018d39b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473d4b0 0x555874747400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f018d39b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555874742620 0x55587474e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f018d39b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555874745620 con 0x55587473cac0 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f018d39b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55587463b5d0 con 0x555874742620 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f018d39b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5558747457a0 con 0x55587473d4b0 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f0188b49640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555874742620 0x55587474e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f018d39b740 10 client.0 _ll_register_callbacks cb 0x555874629080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f0188b49640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555874742620 0x55587474e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:57038/0 (socket says 172.21.15.129:57038) 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f0188b49640 1 -- 172.21.15.129:0/2424260878 learned_addr learned my addr 172.21.15.129:0/2424260878 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f01837fe640 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473d4b0 0x555874747400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f0183fff640 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x555874746ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f01837fe640 1 -- 172.21.15.129:0/2424260878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555874742620 msgr2=0x55587474e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f01837fe640 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555874742620 0x55587474e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f01837fe640 1 -- 172.21.15.129:0/2424260878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 msgr2=0x555874746ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f01837fe640 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x555874746ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f01837fe640 1 -- 172.21.15.129:0/2424260878 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558744dbf70 con 0x55587473d4b0 2024-04-26T21:24:37.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.253+0000 7f0183fff640 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x555874746ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:24:37.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.254+0000 7f01837fe640 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473d4b0 0x555874747400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f01780304b0 tx=0x7f0178000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=271004bd7d0f02a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:37.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.277+0000 7f01677fe640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-26T21:24:37.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.277+0000 7f01677fe640 1 -- 172.21.15.129:0/2424260878 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178046070 con 0x55587473d4b0 2024-04-26T21:24:37.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.277+0000 7f018d39b740 1 -- 172.21.15.129:0/2424260878 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0174013920 con 0x55587473d4b0 2024-04-26T21:24:37.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.277+0000 7f01677fe640 1 -- 172.21.15.129:0/2424260878 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f01780035f0 con 0x55587473d4b0 2024-04-26T21:24:37.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.279+0000 7f01677fe640 1 -- 172.21.15.129:0/2424260878 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f01780037f0 con 0x55587473d4b0 2024-04-26T21:24:37.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.279+0000 7f018d39b740 10 client.25480 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:37.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.279+0000 7f018d39b740 1 -- 172.21.15.129:0/2424260878 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555874373170 con 0x55587473d4b0 2024-04-26T21:24:37.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.282+0000 7f01677fe640 1 -- 172.21.15.129:0/2424260878 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0174013920 con 0x55587473d4b0 2024-04-26T21:24:37.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.282+0000 7f01677fe640 1 -- 172.21.15.129:0/2424260878 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f017803d070 con 0x55587473d4b0 2024-04-26T21:24:37.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.282+0000 7f018d39b740 10 client.25480 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:37.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.282+0000 7f018d39b740 10 client.25480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:37.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.282+0000 7f018d39b740 20 client.25480 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:37.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.282+0000 7f018d39b740 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555874752ea0 0x555874755340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.283+0000 7f018d39b740 4 client.25480 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:24:37.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.283+0000 7f018d39b740 1 -- 172.21.15.129:0/2424260878 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558746af710 con 0x555874752ea0 2024-04-26T21:24:37.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.283+0000 7f0183fff640 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555874752ea0 0x555874755340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.284+0000 7f0183fff640 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555874752ea0 0x555874755340 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c3d58b2ce5a75b server_cookie=3f568fbb9a93849d in_seq=0 out_seq=0 2024-04-26T21:24:37.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.284+0000 7f01677fe640 10 client.25480 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:37.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.289+0000 7f01677fe640 1 -- 172.21.15.129:0/2424260878 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558746af710 con 0x555874752ea0 2024-04-26T21:24:37.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.289+0000 7f01677fe640 10 client.25480 handle_command_reply: tid=0 2024-04-26T21:24:37.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.289+0000 7f018d39b740 1 client.25480 shutdown 2024-04-26T21:24:37.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.293+0000 7f018d39b740 1 -- 172.21.15.129:0/2424260878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473d4b0 msgr2=0x555874747400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:37.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.293+0000 7f018d39b740 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473d4b0 0x555874747400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f01780304b0 tx=0x7f0178000bc0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.295+0000 7f018d39b740 1 -- 172.21.15.129:0/2424260878 shutdown_connections 2024-04-26T21:24:37.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.295+0000 7f018d39b740 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555874752ea0 0x555874755340 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.295+0000 7f018d39b740 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555874742620 0x55587474e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.295+0000 7f018d39b740 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473d4b0 0x555874747400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.295+0000 7f018d39b740 1 --2- 172.21.15.129:0/2424260878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x555874746ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.295+0000 7f018d39b740 1 -- 172.21.15.129:0/2424260878 >> 172.21.15.129:0/2424260878 conn(0x555874738130 msgr2=0x5558747372f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:37.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.299+0000 7f01677fe640 0 client.25480 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:37.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.299+0000 7f018d39b740 1 -- 172.21.15.129:0/2424260878 shutdown_connections 2024-04-26T21:24:37.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.299+0000 7f018d39b740 1 -- 172.21.15.129:0/2424260878 wait complete. 2024-04-26T21:24:37.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.299+0000 7f018d39b740 20 client.25480 trim_cache size 0 max 16384 2024-04-26T21:24:37.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.457+0000 7f018d39b740 1 Processor -- start 2024-04-26T21:24:37.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.457+0000 7f018d39b740 1 -- start start 2024-04-26T21:24:37.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.457+0000 7f018d39b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x55587473e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.458+0000 7f0183fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x55587473e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.458+0000 7f0183fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x55587473e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50728/0 (socket says 172.21.15.129:50728) 2024-04-26T21:24:37.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.458+0000 7f0183fff640 1 -- 172.21.15.129:0/1604065591 learned_addr learned my addr 172.21.15.129:0/1604065591 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:37.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.458+0000 7f018d39b740 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x55587473ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.458+0000 7f018d39b740 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555874742620 0x55587474e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f018d39b740 1 -- 172.21.15.129:0/1604065591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f016800c5d0 con 0x55587473cac0 2024-04-26T21:24:37.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f01837fe640 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x55587473ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f018d39b740 1 -- 172.21.15.129:0/1604065591 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f016800ee50 con 0x55587473d4b0 2024-04-26T21:24:37.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f018d39b740 1 -- 172.21.15.129:0/1604065591 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f016800cc40 con 0x555874742620 2024-04-26T21:24:37.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f0188b49640 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555874742620 0x55587474e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f0183fff640 1 -- 172.21.15.129:0/1604065591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555874742620 msgr2=0x55587474e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:37.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f0183fff640 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555874742620 0x55587474e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f0183fff640 1 -- 172.21.15.129:0/1604065591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 msgr2=0x55587473ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:37.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f0183fff640 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x55587473ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.459+0000 7f0183fff640 1 -- 172.21.15.129:0/1604065591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555874741a60 con 0x55587473cac0 2024-04-26T21:24:37.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.460+0000 7f0183fff640 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x55587473e790 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0174004810 tx=0x7f0174009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399eca6ee97e0f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:37.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.478+0000 7f0171ffb640 1 -- 172.21.15.129:0/1604065591 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017402c070 con 0x55587473cac0 2024-04-26T21:24:37.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.478+0000 7f0171ffb640 1 -- 172.21.15.129:0/1604065591 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0174012ec0 con 0x55587473cac0 2024-04-26T21:24:37.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.478+0000 7f0171ffb640 1 -- 172.21.15.129:0/1604065591 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174017040 con 0x55587473cac0 2024-04-26T21:24:37.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.479+0000 7f018d39b740 1 -- 172.21.15.129:0/1604065591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 msgr2=0x55587473e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:37.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.479+0000 7f018d39b740 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x55587473e790 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0174004810 tx=0x7f0174009110 comp rx=0 tx=0).stop 2024-04-26T21:24:37.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.483+0000 7f018d39b740 1 -- 172.21.15.129:0/1604065591 shutdown_connections 2024-04-26T21:24:37.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.483+0000 7f018d39b740 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x55587473ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.483+0000 7f018d39b740 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555874742620 0x55587474e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.483+0000 7f018d39b740 1 --2- 172.21.15.129:0/1604065591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55587473cac0 0x55587473e790 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.483+0000 7f018d39b740 1 -- 172.21.15.129:0/1604065591 >> 172.21.15.129:0/1604065591 conn(0x555874738130 msgr2=0x555874746ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:37.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.483+0000 7f018d39b740 1 -- 172.21.15.129:0/1604065591 shutdown_connections 2024-04-26T21:24:37.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.483+0000 7f018d39b740 1 -- 172.21.15.129:0/1604065591 wait complete. 2024-04-26T21:24:37.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.484+0000 7f018d39b740 1 -- start start 2024-04-26T21:24:37.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.485+0000 7f018d39b740 1 Processor -- start 2024-04-26T21:24:37.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.485+0000 7f018d39b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 0x55587474c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f018d39b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x55587474c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f018d39b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 0x55587474d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f0183fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 0x55587474c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f0183fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 0x55587474c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34488/0 (socket says 172.21.15.129:34488) 2024-04-26T21:24:37.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f01837fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x55587474c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f0183fff640 1 -- 172.21.15.129:0/3336072834 learned_addr learned my addr 172.21.15.129:0/3336072834 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:37.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f016800d310 con 0x555874742620 2024-04-26T21:24:37.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f016800e780 con 0x55587473d4b0 2024-04-26T21:24:37.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f016800d9e0 con 0x55587473cac0 2024-04-26T21:24:37.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f018d39b740 10 client.0 _ll_register_callbacks cb 0x7f0178030f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:37.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f0188b49640 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 0x55587474d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f01837fe640 1 -- 172.21.15.129:0/3336072834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 msgr2=0x55587474c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:37.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f01837fe640 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 0x55587474c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.486+0000 7f01837fe640 1 -- 172.21.15.129:0/3336072834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 msgr2=0x55587474d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:37.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.487+0000 7f01837fe640 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 0x55587474d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:37.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.487+0000 7f01837fe640 1 -- 172.21.15.129:0/3336072834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587474db90 con 0x55587473d4b0 2024-04-26T21:24:37.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.487+0000 7f0188b49640 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 0x55587474d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:24:37.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.487+0000 7f01837fe640 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x55587474c590 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0178009780 tx=0x7f017802ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c46cb15ae62feda1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:37.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.488+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0174012ec0 con 0x55587473d4b0 2024-04-26T21:24:37.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.526+0000 7f01677fe640 10 client.25562 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:24:37.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.526+0000 7f01677fe640 1 -- 172.21.15.129:0/3336072834 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178046040 con 0x55587473d4b0 2024-04-26T21:24:37.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.526+0000 7f01677fe640 1 -- 172.21.15.129:0/3336072834 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f01780035f0 con 0x55587473d4b0 2024-04-26T21:24:37.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.526+0000 7f01677fe640 1 -- 172.21.15.129:0/3336072834 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0178003ba0 con 0x55587473d4b0 2024-04-26T21:24:37.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.527+0000 7f018d39b740 10 client.25562 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:37.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.527+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({fsmap=19}) -- 0x5558747b9910 con 0x55587473d4b0 2024-04-26T21:24:37.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.535+0000 7f01677fe640 1 -- 172.21.15.129:0/3336072834 <== mon.2 v2:172.21.15.150:3300/0 4 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f017804d560 con 0x55587473d4b0 2024-04-26T21:24:37.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.536+0000 7f018d39b740 10 client.25562 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:37.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.536+0000 7f018d39b740 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:37.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.536+0000 7f018d39b740 20 client.25562 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:37.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.536+0000 7f018d39b740 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5558747ba120 0x5558747bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:37.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.536+0000 7f018d39b740 4 client.25562 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:24:37.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.536+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f0174012ec0 con 0x5558747ba120 2024-04-26T21:24:37.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.536+0000 7f0183fff640 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5558747ba120 0x5558747bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:37.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.537+0000 7f0183fff640 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5558747ba120 0x5558747bc540 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df895b85e79113a server_cookie=23a0af46862465b9 in_seq=0 out_seq=0 2024-04-26T21:24:37.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:37.537+0000 7f01677fe640 10 client.25562 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:37.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:37 smithi150 ceph-mon[32290]: pgmap v549: 97 pgs: 97 active+clean; 806 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 485 op/s 2024-04-26T21:24:37.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:37 smithi129 ceph-mon[29237]: pgmap v549: 97 pgs: 97 active+clean; 806 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 485 op/s 2024-04-26T21:24:37.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:37 smithi171 ceph-mon[33478]: pgmap v549: 97 pgs: 97 active+clean; 806 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 485 op/s 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f01677fe640 1 -- 172.21.15.129:0/3336072834 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f0174012ec0 con 0x5558747ba120 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f01677fe640 10 client.25562 handle_command_reply: tid=0 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f018d39b740 1 client.25562 shutdown 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 msgr2=0x55587474c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f018d39b740 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x55587474c590 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0178009780 tx=0x7f017802ac80 comp rx=0 tx=0).stop 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 shutdown_connections 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f018d39b740 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5558747ba120 0x5558747bc540 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f018d39b740 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55587473d4b0 0x55587474c590 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f018d39b740 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555874742620 0x55587474d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f018d39b740 1 --2- 172.21.15.129:0/3336072834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55587473cac0 0x55587474c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 >> 172.21.15.129:0/3336072834 conn(0x555874738130 msgr2=0x555874746e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:38.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.340+0000 7f01677fe640 0 client.25562 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:38.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.341+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 shutdown_connections 2024-04-26T21:24:38.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.341+0000 7f018d39b740 1 -- 172.21.15.129:0/3336072834 wait complete. 2024-04-26T21:24:38.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.341+0000 7f018d39b740 20 client.25562 trim_cache size 0 max 16384 2024-04-26T21:24:38.344 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:24:38.344 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:24:38.344 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "952ef762-6180-47c4-829b-d71b70ddd250", 2024-04-26T21:24:38.344 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:24:38.344 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:24:38.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 -- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f016c0a3e70 msgr2=0x7f016c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:38.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f016c0a3e70 0x7f016c0a6330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0178005b60 tx=0x7f017802c040 comp rx=0 tx=0).stop 2024-04-26T21:24:38.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 -- 172.21.15.129:0/3330353594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f018410a8d0 msgr2=0x7f01841aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:38.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f018410a8d0 0x7f01841aedc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f017c00eda0 tx=0x7f017c00c6e0 comp rx=0 tx=0).stop 2024-04-26T21:24:38.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 -- 172.21.15.129:0/3330353594 shutdown_connections 2024-04-26T21:24:38.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f016c0a3e70 0x7f016c0a6330 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:38.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184109f10 0x7f01841a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:38.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f018410a8d0 0x7f01841aedc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:38.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 --2- 172.21.15.129:0/3330353594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0184073c10 0x7f01841a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:38.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.377+0000 7f018934a640 1 -- 172.21.15.129:0/3330353594 >> 172.21.15.129:0/3330353594 conn(0x7f018406f780 msgr2=0x7f01840713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:38.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.383+0000 7f018934a640 1 -- 172.21.15.129:0/3330353594 shutdown_connections 2024-04-26T21:24:38.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:38.383+0000 7f018934a640 1 -- 172.21.15.129:0/3330353594 wait complete. 2024-04-26T21:24:38.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:24:38.499 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:24:39.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:39 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:24:39.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:39 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:24:39.471 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:39 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:24:39.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:39 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:24:39.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:39 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:24:39.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:39 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:24:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:39 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:24:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:39 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:24:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:39 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:24:40.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:40 smithi129 ceph-mon[29237]: pgmap v550: 97 pgs: 97 active+clean; 808 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 411 op/s 2024-04-26T21:24:40.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:40 smithi171 ceph-mon[33478]: pgmap v550: 97 pgs: 97 active+clean; 808 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 411 op/s 2024-04-26T21:24:40.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:40 smithi150 ceph-mon[32290]: pgmap v550: 97 pgs: 97 active+clean; 808 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 411 op/s 2024-04-26T21:24:41.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.182+0000 7f57dd2c9640 1 -- 172.21.15.129:0/3594068296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8109f10 msgr2=0x7f57d810a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:41.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.182+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/3594068296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8109f10 0x7f57d810a390 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009900 tx=0x7f57cc02a400 comp rx=0 tx=0).stop 2024-04-26T21:24:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.186+0000 7f57dd2c9640 1 -- 172.21.15.129:0/3594068296 shutdown_connections 2024-04-26T21:24:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.186+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/3594068296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d810a8d0 0x7f57d810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.186+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/3594068296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8109f10 0x7f57d810a390 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.186+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/3594068296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8073c10 0x7f57d8074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.186+0000 7f57dd2c9640 1 -- 172.21.15.129:0/3594068296 >> 172.21.15.129:0/3594068296 conn(0x7f57d806f780 msgr2=0x7f57d8071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:41.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.194+0000 7f57dd2c9640 1 -- 172.21.15.129:0/3594068296 shutdown_connections 2024-04-26T21:24:41.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.194+0000 7f57dd2c9640 1 -- 172.21.15.129:0/3594068296 wait complete. 2024-04-26T21:24:41.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.195+0000 7f57dd2c9640 1 Processor -- start 2024-04-26T21:24:41.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.195+0000 7f57dd2c9640 1 -- start start 2024-04-26T21:24:41.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.195+0000 7f57dd2c9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8073c10 0x7f57d81180b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.195+0000 7f57dd2c9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8109f10 0x7f57d81185f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.196+0000 7f57dd2c9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d810a8d0 0x7f57d81b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.196+0000 7f57dd2c9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57d811d110 con 0x7f57d8109f10 2024-04-26T21:24:41.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.196+0000 7f57dd2c9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57d811cf90 con 0x7f57d8073c10 2024-04-26T21:24:41.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.196+0000 7f57dd2c9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f57d811d290 con 0x7f57d810a8d0 2024-04-26T21:24:41.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.196+0000 7f57dcac8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d810a8d0 0x7f57d81b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.196+0000 7f57dcac8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d810a8d0 0x7f57d81b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34510/0 (socket says 172.21.15.129:34510) 2024-04-26T21:24:41.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.196+0000 7f57dcac8640 1 -- 172.21.15.129:0/1075452937 learned_addr learned my addr 172.21.15.129:0/1075452937 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:41.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.196+0000 7f57d7fff640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8073c10 0x7f57d81180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.198+0000 7f57d7fff640 1 -- 172.21.15.129:0/1075452937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d810a8d0 msgr2=0x7f57d81b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:41.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.198+0000 7f57d7fff640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d810a8d0 0x7f57d81b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.198+0000 7f57d7fff640 1 -- 172.21.15.129:0/1075452937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8109f10 msgr2=0x7f57d81185f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:41.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.198+0000 7f57d7fff640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8109f10 0x7f57d81185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.198+0000 7f57d7fff640 1 -- 172.21.15.129:0/1075452937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d81b7d40 con 0x7f57d8073c10 2024-04-26T21:24:41.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.198+0000 7f57dcac8640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d810a8d0 0x7f57d81b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:24:41.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.199+0000 7f57d7fff640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8073c10 0x7f57d81180b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f57c800d6d0 tx=0x7f57c800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:41.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.236+0000 7f57d57fa640 1 -- 172.21.15.129:0/1075452937 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8017070 con 0x7f57d8073c10 2024-04-26T21:24:41.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.236+0000 7f57d57fa640 1 -- 172.21.15.129:0/1075452937 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57c8009820 con 0x7f57d8073c10 2024-04-26T21:24:41.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.236+0000 7f57dd2c9640 1 -- 172.21.15.129:0/1075452937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d81b7f70 con 0x7f57d8073c10 2024-04-26T21:24:41.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.236+0000 7f57dd2c9640 1 -- 172.21.15.129:0/1075452937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d81b83e0 con 0x7f57d8073c10 2024-04-26T21:24:41.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57e1247740 1 Processor -- start 2024-04-26T21:24:41.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57e1247740 1 -- start start 2024-04-26T21:24:41.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57e1247740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 0x56362a4c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57e1247740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4c6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57e1247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6c40 0x56362a4cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57e1247740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56362a429d80 con 0x56362a4c50b0 2024-04-26T21:24:41.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57e1247740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56362a3e38b0 con 0x56362a4c6280 2024-04-26T21:24:41.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57e1247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56362a425d40 con 0x56362a4c6c40 2024-04-26T21:24:41.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57d57fa640 1 -- 172.21.15.129:0/1075452937 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8004c40 con 0x7f57d8073c10 2024-04-26T21:24:41.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.240+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6c40 0x56362a4cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.241+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6c40 0x56362a4cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34514/0 (socket says 172.21.15.129:34514) 2024-04-26T21:24:41.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.241+0000 7f57d77fe640 1 -- 172.21.15.129:0/1524145356 learned_addr learned my addr 172.21.15.129:0/1524145356 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:41.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.241+0000 7f57dcac8640 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 0x56362a4c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.242+0000 7f57d7fff640 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.242+0000 7f57d77fe640 1 -- 172.21.15.129:0/1524145356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 msgr2=0x56362a4c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:41.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.242+0000 7f57d77fe640 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.242+0000 7f57d77fe640 1 -- 172.21.15.129:0/1524145356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 msgr2=0x56362a4c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:41.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.242+0000 7f57d77fe640 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 0x56362a4c54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.248+0000 7f57d77fe640 1 -- 172.21.15.129:0/1524145356 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56362a3b4a40 con 0x56362a4c6c40 2024-04-26T21:24:41.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.248+0000 7f57d7fff640 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:24:41.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.248+0000 7f57d77fe640 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6c40 0x56362a4cd4d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009850 tx=0x7f57cc005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96790de771476c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:41.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.250+0000 7f57d57fa640 1 -- 172.21.15.129:0/1075452937 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f57c8010670 con 0x7f57d8073c10 2024-04-26T21:24:41.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.251+0000 7f57d57fa640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c00a3dc0 0x7f57c00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.252+0000 7f57d57fa640 1 -- 172.21.15.129:0/1075452937 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f57c80d00c0 con 0x7f57d8073c10 2024-04-26T21:24:41.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.252+0000 7f57dcac8640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c00a3dc0 0x7f57c00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.254+0000 7f57dcac8640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c00a3dc0 0x7f57c00a6280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x56362a4c6180 tx=0x7f57d0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:41.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.256+0000 7f57be7fc640 1 -- 172.21.15.129:0/1524145356 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc03e030 con 0x56362a4c6c40 2024-04-26T21:24:41.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.256+0000 7f57be7fc640 1 -- 172.21.15.129:0/1524145356 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57cc004020 con 0x56362a4c6c40 2024-04-26T21:24:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.259+0000 7f57e1247740 1 -- 172.21.15.129:0/1524145356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6c40 msgr2=0x56362a4cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.259+0000 7f57e1247740 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6c40 0x56362a4cd4d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009850 tx=0x7f57cc005be0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.259+0000 7f57e1247740 1 -- 172.21.15.129:0/1524145356 shutdown_connections 2024-04-26T21:24:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.261+0000 7f57e1247740 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.261+0000 7f57e1247740 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6c40 0x56362a4cd4d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.261+0000 7f57e1247740 1 --2- 172.21.15.129:0/1524145356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 0x56362a4c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.261+0000 7f57e1247740 1 -- 172.21.15.129:0/1524145356 >> 172.21.15.129:0/1524145356 conn(0x56362a4c05a0 msgr2=0x56362a4c09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:41.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.265+0000 7f57e1247740 1 -- 172.21.15.129:0/1524145356 shutdown_connections 2024-04-26T21:24:41.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.265+0000 7f57e1247740 1 -- 172.21.15.129:0/1524145356 wait complete. 2024-04-26T21:24:41.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.265+0000 7f57e1247740 1 -- start start 2024-04-26T21:24:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.266+0000 7f57e1247740 1 Processor -- start 2024-04-26T21:24:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.266+0000 7f57e1247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 0x56362a4cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.266+0000 7f57e1247740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.266+0000 7f57e1247740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c6c40 0x56362a4d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.266+0000 7f57e1247740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56362a4cdc40 con 0x56362a4c6c40 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.266+0000 7f57e1247740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56362a3b0d10 con 0x56362a4c6280 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.266+0000 7f57e1247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56362a4cddc0 con 0x56362a4c50b0 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.266+0000 7f57e1247740 10 client.0 _ll_register_callbacks cb 0x56362a3dd210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.266+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.267+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:57086/0 (socket says 172.21.15.129:57086) 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.267+0000 7f57d77fe640 1 -- 172.21.15.129:0/2431822217 learned_addr learned my addr 172.21.15.129:0/2431822217 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.267+0000 7f57dcac8640 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c6c40 0x56362a4d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.267+0000 7f57d7fff640 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 0x56362a4cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.267+0000 7f57d77fe640 1 -- 172.21.15.129:0/2431822217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 msgr2=0x56362a4cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.267+0000 7f57d77fe640 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 0x56362a4cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.267+0000 7f57d77fe640 1 -- 172.21.15.129:0/2431822217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c6c40 msgr2=0x56362a4d6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.267+0000 7f57d77fe640 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c6c40 0x56362a4d6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.267+0000 7f57d77fe640 1 -- 172.21.15.129:0/2431822217 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56362a2057d0 con 0x56362a4c6280 2024-04-26T21:24:41.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.268+0000 7f57d77fe640 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4cfbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f57cc02a9f0 tx=0x7f57cc004fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab0cc0aaf85ffce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:41.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.282+0000 7f57b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:24:41.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.282+0000 7f57b77fe640 1 -- 172.21.15.129:0/2431822217 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc03e050 con 0x56362a4c6280 2024-04-26T21:24:41.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.282+0000 7f57b77fe640 1 -- 172.21.15.129:0/2431822217 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f57cc005170 con 0x56362a4c6280 2024-04-26T21:24:41.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.282+0000 7f57e1247740 1 -- 172.21.15.129:0/2431822217 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56362a437990 con 0x56362a4c6280 2024-04-26T21:24:41.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.282+0000 7f57b77fe640 1 -- 172.21.15.129:0/2431822217 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f57cc005370 con 0x56362a4c6280 2024-04-26T21:24:41.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.283+0000 7f57e1247740 10 client.25571 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:41.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.283+0000 7f57e1247740 1 -- 172.21.15.129:0/2431822217 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56362a13b3f0 con 0x56362a4c6280 2024-04-26T21:24:41.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.288+0000 7f57b77fe640 1 -- 172.21.15.129:0/2431822217 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56362a437990 con 0x56362a4c6280 2024-04-26T21:24:41.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.288+0000 7f57b77fe640 1 -- 172.21.15.129:0/2431822217 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f57cc043570 con 0x56362a4c6280 2024-04-26T21:24:41.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.288+0000 7f57e1247740 10 client.25571 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:41.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.288+0000 7f57e1247740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:41.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.288+0000 7f57e1247740 20 client.25571 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:41.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.288+0000 7f57e1247740 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56362a4db230 0x56362a4dd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.288+0000 7f57e1247740 4 client.25571 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:24:41.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.288+0000 7f57e1247740 1 -- 172.21.15.129:0/2431822217 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56362a431800 con 0x56362a4db230 2024-04-26T21:24:41.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.288+0000 7f57d7fff640 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56362a4db230 0x56362a4dd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.289+0000 7f57d7fff640 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56362a4db230 0x56362a4dd6d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a718fb6115de05a1 server_cookie=528fd281764c7938 in_seq=0 out_seq=0 2024-04-26T21:24:41.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.289+0000 7f57b77fe640 10 client.25571 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.291+0000 7f57b77fe640 1 -- 172.21.15.129:0/2431822217 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56362a431800 con 0x56362a4db230 2024-04-26T21:24:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.291+0000 7f57b77fe640 10 client.25571 handle_command_reply: tid=0 2024-04-26T21:24:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.291+0000 7f57e1247740 1 client.25571 shutdown 2024-04-26T21:24:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.299+0000 7f57e1247740 1 -- 172.21.15.129:0/2431822217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 msgr2=0x56362a4cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.299+0000 7f57e1247740 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4cfbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f57cc02a9f0 tx=0x7f57cc004fb0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.300+0000 7f57e1247740 1 -- 172.21.15.129:0/2431822217 shutdown_connections 2024-04-26T21:24:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.300+0000 7f57e1247740 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56362a4db230 0x56362a4dd6d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.300+0000 7f57e1247740 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6280 0x56362a4cfbb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.300+0000 7f57e1247740 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c6c40 0x56362a4d6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.300+0000 7f57e1247740 1 --2- 172.21.15.129:0/2431822217 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 0x56362a4cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.300+0000 7f57b77fe640 0 client.25571 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.300+0000 7f57e1247740 1 -- 172.21.15.129:0/2431822217 >> 172.21.15.129:0/2431822217 conn(0x56362a4c05a0 msgr2=0x56362a4bf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:41.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.302+0000 7f57e1247740 1 -- 172.21.15.129:0/2431822217 shutdown_connections 2024-04-26T21:24:41.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.302+0000 7f57e1247740 1 -- 172.21.15.129:0/2431822217 wait complete. 2024-04-26T21:24:41.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.302+0000 7f57e1247740 20 client.25571 trim_cache size 0 max 16384 2024-04-26T21:24:41.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.356+0000 7f57e1247740 1 Processor -- start 2024-04-26T21:24:41.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.356+0000 7f57e1247740 1 -- start start 2024-04-26T21:24:41.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.356+0000 7f57e1247740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 0x56362a4cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.357+0000 7f57e1247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6280 0x56362a4d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.357+0000 7f57e1247740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.357+0000 7f57e1247740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57c400c5f0 con 0x56362a4c50b0 2024-04-26T21:24:41.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.357+0000 7f57e1247740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57c400ee70 con 0x56362a4c6c40 2024-04-26T21:24:41.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.357+0000 7f57e1247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f57c400cc60 con 0x56362a4c6280 2024-04-26T21:24:41.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57d7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 0x56362a4cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57d7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 0x56362a4cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50780/0 (socket says 172.21.15.129:50780) 2024-04-26T21:24:41.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57d7fff640 1 -- 172.21.15.129:0/1783109333 learned_addr learned my addr 172.21.15.129:0/1783109333 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:41.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57dcac8640 1 --2- 172.21.15.129:0/1783109333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57d77fe640 1 --2- 172.21.15.129:0/1783109333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6280 0x56362a4d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57dcac8640 1 -- 172.21.15.129:0/1783109333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6280 msgr2=0x56362a4d6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:41.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57dcac8640 1 --2- 172.21.15.129:0/1783109333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6280 0x56362a4d6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57dcac8640 1 -- 172.21.15.129:0/1783109333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 msgr2=0x56362a4cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:41.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57dcac8640 1 --2- 172.21.15.129:0/1783109333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 0x56362a4cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.358+0000 7f57dcac8640 1 -- 172.21.15.129:0/1783109333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56362a0fb0f0 con 0x56362a4c6c40 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.359+0000 7f57dcac8640 1 --2- 172.21.15.129:0/1783109333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4c3280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f57d0011740 tx=0x7f57d0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5bb2b870f6415bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.359+0000 7f57bdffb640 1 -- 172.21.15.129:0/1783109333 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d002d070 con 0x56362a4c6c40 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.359+0000 7f57bdffb640 1 -- 172.21.15.129:0/1783109333 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d001f0d0 con 0x56362a4c6c40 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.360+0000 7f57e1247740 1 -- 172.21.15.129:0/1783109333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 msgr2=0x56362a4c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.360+0000 7f57e1247740 1 --2- 172.21.15.129:0/1783109333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4c3280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f57d0011740 tx=0x7f57d0017000 comp rx=0 tx=0).stop 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.361+0000 7f57e1247740 1 -- 172.21.15.129:0/1783109333 shutdown_connections 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.361+0000 7f57e1247740 1 --2- 172.21.15.129:0/1783109333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4c3280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.361+0000 7f57e1247740 1 --2- 172.21.15.129:0/1783109333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c6280 0x56362a4d6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.361+0000 7f57e1247740 1 --2- 172.21.15.129:0/1783109333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a4c50b0 0x56362a4cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.361+0000 7f57e1247740 1 -- 172.21.15.129:0/1783109333 >> 172.21.15.129:0/1783109333 conn(0x56362a4c05a0 msgr2=0x56362a4ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.373+0000 7f57e1247740 1 -- 172.21.15.129:0/1783109333 shutdown_connections 2024-04-26T21:24:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.373+0000 7f57e1247740 1 -- 172.21.15.129:0/1783109333 wait complete. 2024-04-26T21:24:41.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.373+0000 7f57e1247740 1 -- start start 2024-04-26T21:24:41.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.374+0000 7f57e1247740 1 Processor -- start 2024-04-26T21:24:41.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.374+0000 7f57e1247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 0x56362a4d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.374+0000 7f57e1247740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.375+0000 7f57e1247740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a53d830 0x56362a53fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.375+0000 7f57e1247740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57c400d330 con 0x56362a53d830 2024-04-26T21:24:41.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.375+0000 7f57e1247740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57c400e7a0 con 0x56362a4c6c40 2024-04-26T21:24:41.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.375+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.375+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:57100/0 (socket says 172.21.15.129:57100) 2024-04-26T21:24:41.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.375+0000 7f57d77fe640 1 -- 172.21.15.129:0/3825094740 learned_addr learned my addr 172.21.15.129:0/3825094740 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:24:41.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.375+0000 7f57d7fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 0x56362a4d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.375+0000 7f57e1247740 1 -- 172.21.15.129:0/3825094740 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f57c400da00 con 0x56362a4c50b0 2024-04-26T21:24:41.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.375+0000 7f57e1247740 10 client.0 _ll_register_callbacks cb 0x7f57d0009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:24:41.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.377+0000 7f57d77fe640 1 -- 172.21.15.129:0/3825094740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 msgr2=0x56362a4d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:41.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.377+0000 7f57d77fe640 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 0x56362a4d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.377+0000 7f57d77fe640 1 -- 172.21.15.129:0/3825094740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a53d830 msgr2=0x56362a53fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:24:41.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.377+0000 7f57d77fe640 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a53d830 0x56362a53fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.377+0000 7f57d77fe640 1 -- 172.21.15.129:0/3825094740 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56362a3ea4b0 con 0x56362a4c6c40 2024-04-26T21:24:41.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.377+0000 7f57d77fe640 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4d4130 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57cc02a9c0 tx=0x7f57cc02ab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b00c0412099124f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:41.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.377+0000 7f57b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:24:41.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.377+0000 7f57b77fe640 1 -- 172.21.15.129:0/3825094740 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc03e040 con 0x56362a4c6c40 2024-04-26T21:24:41.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.378+0000 7f57e1247740 1 -- 172.21.15.129:0/3825094740 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d001f0d0 con 0x56362a4c6c40 2024-04-26T21:24:41.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.378+0000 7f57b77fe640 1 -- 172.21.15.129:0/3825094740 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f57cc005170 con 0x56362a4c6c40 2024-04-26T21:24:41.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.378+0000 7f57b77fe640 1 -- 172.21.15.129:0/3825094740 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f57cc043670 con 0x56362a4c6c40 2024-04-26T21:24:41.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.387+0000 7f57e1247740 10 client.25577 fetch_fsmap learned FSMap version 19 2024-04-26T21:24:41.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.387+0000 7f57e1247740 1 -- 172.21.15.129:0/3825094740 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56362a541320 con 0x56362a4c6c40 2024-04-26T21:24:41.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.404+0000 7f57b77fe640 1 -- 172.21.15.129:0/3825094740 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d001f0d0 con 0x56362a4c6c40 2024-04-26T21:24:41.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.404+0000 7f57b77fe640 1 -- 172.21.15.129:0/3825094740 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f57cc003ae0 con 0x56362a4c6c40 2024-04-26T21:24:41.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.404+0000 7f57e1247740 10 client.25577 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:24:41.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.404+0000 7f57e1247740 10 client.25577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:24:41.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.404+0000 7f57e1247740 20 client.25577 populate_metadata read hostname 'smithi129' 2024-04-26T21:24:41.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.404+0000 7f57e1247740 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56362a541ac0 0x56362a543ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:41.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.405+0000 7f57e1247740 4 client.25577 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:24:41.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.405+0000 7f57e1247740 1 -- 172.21.15.129:0/3825094740 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56362a3ac4a0 con 0x56362a541ac0 2024-04-26T21:24:41.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.405+0000 7f57d7fff640 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56362a541ac0 0x56362a543ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:41.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.405+0000 7f57d7fff640 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56362a541ac0 0x56362a543ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268cd2acb483d634 server_cookie=27bca27583a9c286 in_seq=0 out_seq=0 2024-04-26T21:24:41.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.405+0000 7f57b77fe640 10 client.25577 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:41.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.407+0000 7f57b77fe640 1 -- 172.21.15.129:0/3825094740 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x56362a3ac4a0 con 0x56362a541ac0 2024-04-26T21:24:41.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.407+0000 7f57b77fe640 10 client.25577 handle_command_reply: tid=0 2024-04-26T21:24:41.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.407+0000 7f57e1247740 1 client.25577 shutdown 2024-04-26T21:24:41.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.408+0000 7f57e1247740 1 -- 172.21.15.129:0/3825094740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 msgr2=0x56362a4d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:41.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.408+0000 7f57e1247740 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4d4130 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57cc02a9c0 tx=0x7f57cc02ab80 comp rx=0 tx=0).stop 2024-04-26T21:24:41.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.411+0000 7f57e1247740 1 -- 172.21.15.129:0/3825094740 shutdown_connections 2024-04-26T21:24:41.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.411+0000 7f57e1247740 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56362a541ac0 0x56362a543ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.411+0000 7f57e1247740 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56362a4c6c40 0x56362a4d4130 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.411+0000 7f57e1247740 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56362a53d830 0x56362a53fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.411+0000 7f57e1247740 1 --2- 172.21.15.129:0/3825094740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56362a4c50b0 0x56362a4d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.411+0000 7f57e1247740 1 -- 172.21.15.129:0/3825094740 >> 172.21.15.129:0/3825094740 conn(0x56362a4c05a0 msgr2=0x56362a4bee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:41.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.413+0000 7f57b77fe640 0 client.25577 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:24:41.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.436+0000 7f57e1247740 1 -- 172.21.15.129:0/3825094740 shutdown_connections 2024-04-26T21:24:41.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.436+0000 7f57e1247740 1 -- 172.21.15.129:0/3825094740 wait complete. 2024-04-26T21:24:41.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.436+0000 7f57e1247740 20 client.25577 trim_cache size 0 max 16384 2024-04-26T21:24:41.441 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:24:41.441 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (22 inodes in the stack)", 2024-04-26T21:24:41.441 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:24:41.441 INFO:teuthology.orchestra.run.smithi129.stdout: "952ef762-6180-47c4-829b-d71b70ddd250": { 2024-04-26T21:24:41.441 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:24:41.441 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "952ef762-6180-47c4-829b-d71b70ddd250", 2024-04-26T21:24:41.442 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:24:41.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:24:41.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:24:41.442 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:24:41.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 -- 172.21.15.129:0/1075452937 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c00a3dc0 msgr2=0x7f57c00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:41.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c00a3dc0 0x7f57c00a6280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x56362a4c6180 tx=0x7f57d0009500 comp rx=0 tx=0).stop 2024-04-26T21:24:41.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 -- 172.21.15.129:0/1075452937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8073c10 msgr2=0x7f57d81180b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:41.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8073c10 0x7f57d81180b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f57c800d6d0 tx=0x7f57c800db90 comp rx=0 tx=0).stop 2024-04-26T21:24:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 -- 172.21.15.129:0/1075452937 shutdown_connections 2024-04-26T21:24:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f57c00a3dc0 0x7f57c00a6280 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d810a8d0 0x7f57d81b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d8109f10 0x7f57d81185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 --2- 172.21.15.129:0/1075452937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d8073c10 0x7f57d81180b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:41.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.442+0000 7f57dd2c9640 1 -- 172.21.15.129:0/1075452937 >> 172.21.15.129:0/1075452937 conn(0x7f57d806f780 msgr2=0x7f57d810d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:41.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.447+0000 7f57dd2c9640 1 -- 172.21.15.129:0/1075452937 shutdown_connections 2024-04-26T21:24:41.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:24:41.448+0000 7f57dd2c9640 1 -- 172.21.15.129:0/1075452937 wait complete. 2024-04-26T21:24:41.504 INFO:tasks.cephfs.filesystem:scrub status for tag:952ef762-6180-47c4-829b-d71b70ddd250 - {'path': '/', 'tag': '952ef762-6180-47c4-829b-d71b70ddd250', 'options': 'recursive,force'} 2024-04-26T21:24:41.789 INFO:tasks.workunit.client.0.smithi129.stdout: 181 1532803 336 1051106 375 854899 1161 2024-04-26T21:24:41.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:41 smithi150 ceph-mon[32290]: pgmap v551: 97 pgs: 97 active+clean; 822 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 431 op/s 2024-04-26T21:24:41.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:41 smithi129 ceph-mon[29237]: pgmap v551: 97 pgs: 97 active+clean; 822 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 431 op/s 2024-04-26T21:24:41.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:41 smithi171 ceph-mon[33478]: pgmap v551: 97 pgs: 97 active+clean; 822 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 431 op/s 2024-04-26T21:24:43.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:43 smithi150 ceph-mon[32290]: pgmap v552: 97 pgs: 97 active+clean; 855 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 549 op/s 2024-04-26T21:24:43.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:43 smithi129 ceph-mon[29237]: pgmap v552: 97 pgs: 97 active+clean; 855 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 549 op/s 2024-04-26T21:24:43.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:43 smithi171 ceph-mon[33478]: pgmap v552: 97 pgs: 97 active+clean; 855 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 549 op/s 2024-04-26T21:24:45.582 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-26T21:24:45.583 INFO:tasks.workunit.client.0.smithi129.stdout:Final score for writes: 181 2024-04-26T21:24:45.583 INFO:tasks.workunit.client.0.smithi129.stdout:Final score for reads : 889787 2024-04-26T21:24:45.583 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-26T21:24:45.583 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T21:24:45.583 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T21:24:45.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:45 smithi171 ceph-mon[33478]: pgmap v553: 97 pgs: 97 active+clean; 864 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.2 MiB/s wr, 545 op/s 2024-04-26T21:24:45.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:45 smithi150 ceph-mon[32290]: pgmap v553: 97 pgs: 97 active+clean; 864 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.2 MiB/s wr, 545 op/s 2024-04-26T21:24:45.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:45 smithi129 ceph-mon[29237]: pgmap v553: 97 pgs: 97 active+clean; 864 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.2 MiB/s wr, 545 op/s 2024-04-26T21:24:47.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:47 smithi150 ceph-mon[32290]: pgmap v554: 97 pgs: 97 active+clean; 860 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 634 op/s 2024-04-26T21:24:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:47 smithi129 ceph-mon[29237]: pgmap v554: 97 pgs: 97 active+clean; 860 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 634 op/s 2024-04-26T21:24:47.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:47 smithi171 ceph-mon[33478]: pgmap v554: 97 pgs: 97 active+clean; 860 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 634 op/s 2024-04-26T21:24:49.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:49 smithi150 ceph-mon[32290]: pgmap v555: 97 pgs: 97 active+clean; 843 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.6 MiB/s wr, 694 op/s 2024-04-26T21:24:49.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:49 smithi129 ceph-mon[29237]: pgmap v555: 97 pgs: 97 active+clean; 843 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.6 MiB/s wr, 694 op/s 2024-04-26T21:24:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:49 smithi171 ceph-mon[33478]: pgmap v555: 97 pgs: 97 active+clean; 843 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.6 MiB/s wr, 694 op/s 2024-04-26T21:24:51.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:51 smithi150 ceph-mon[32290]: pgmap v556: 97 pgs: 97 active+clean; 840 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.8 MiB/s wr, 807 op/s 2024-04-26T21:24:51.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:51 smithi129 ceph-mon[29237]: pgmap v556: 97 pgs: 97 active+clean; 840 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.8 MiB/s wr, 807 op/s 2024-04-26T21:24:51.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:51 smithi171 ceph-mon[33478]: pgmap v556: 97 pgs: 97 active+clean; 840 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.8 MiB/s wr, 807 op/s 2024-04-26T21:24:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:53 smithi150 ceph-mon[32290]: pgmap v557: 97 pgs: 97 active+clean; 825 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 910 op/s 2024-04-26T21:24:53.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:53 smithi129 ceph-mon[29237]: pgmap v557: 97 pgs: 97 active+clean; 825 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 910 op/s 2024-04-26T21:24:53.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:53 smithi171 ceph-mon[33478]: pgmap v557: 97 pgs: 97 active+clean; 825 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 910 op/s 2024-04-26T21:24:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:55 smithi150 ceph-mon[32290]: pgmap v558: 97 pgs: 97 active+clean; 821 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 773 op/s 2024-04-26T21:24:55.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:55 smithi129 ceph-mon[29237]: pgmap v558: 97 pgs: 97 active+clean; 821 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 773 op/s 2024-04-26T21:24:55.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:55 smithi171 ceph-mon[33478]: pgmap v558: 97 pgs: 97 active+clean; 821 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 773 op/s 2024-04-26T21:24:57.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:57 smithi150 ceph-mon[32290]: pgmap v559: 97 pgs: 97 active+clean; 809 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 903 op/s 2024-04-26T21:24:57.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:57 smithi129 ceph-mon[29237]: pgmap v559: 97 pgs: 97 active+clean; 809 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 903 op/s 2024-04-26T21:24:57.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:57 smithi171 ceph-mon[33478]: pgmap v559: 97 pgs: 97 active+clean; 809 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 903 op/s 2024-04-26T21:24:59.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:24:59 smithi150 ceph-mon[32290]: pgmap v560: 97 pgs: 97 active+clean; 800 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 971 op/s 2024-04-26T21:24:59.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:24:59 smithi129 ceph-mon[29237]: pgmap v560: 97 pgs: 97 active+clean; 800 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 971 op/s 2024-04-26T21:24:59.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:24:59 smithi171 ceph-mon[33478]: pgmap v560: 97 pgs: 97 active+clean; 800 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 971 op/s 2024-04-26T21:25:01.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:01 smithi150 ceph-mon[32290]: pgmap v561: 97 pgs: 97 active+clean; 789 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 1.05k op/s 2024-04-26T21:25:01.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:01 smithi129 ceph-mon[29237]: pgmap v561: 97 pgs: 97 active+clean; 789 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 1.05k op/s 2024-04-26T21:25:01.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:01 smithi171 ceph-mon[33478]: pgmap v561: 97 pgs: 97 active+clean; 789 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 1.05k op/s 2024-04-26T21:25:03.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:03 smithi150 ceph-mon[32290]: pgmap v562: 97 pgs: 97 active+clean; 759 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 1.08k op/s 2024-04-26T21:25:03.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:03 smithi129 ceph-mon[29237]: pgmap v562: 97 pgs: 97 active+clean; 759 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 1.08k op/s 2024-04-26T21:25:03.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:03 smithi171 ceph-mon[33478]: pgmap v562: 97 pgs: 97 active+clean; 759 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 1.08k op/s 2024-04-26T21:25:05.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:05 smithi150 ceph-mon[32290]: pgmap v563: 97 pgs: 97 active+clean; 756 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 931 op/s 2024-04-26T21:25:05.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:05 smithi129 ceph-mon[29237]: pgmap v563: 97 pgs: 97 active+clean; 756 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 931 op/s 2024-04-26T21:25:05.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:05 smithi171 ceph-mon[33478]: pgmap v563: 97 pgs: 97 active+clean; 756 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 931 op/s 2024-04-26T21:25:07.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:07 smithi150 ceph-mon[32290]: pgmap v564: 97 pgs: 97 active+clean; 734 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 1.01k op/s 2024-04-26T21:25:07.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:07 smithi129 ceph-mon[29237]: pgmap v564: 97 pgs: 97 active+clean; 734 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 1.01k op/s 2024-04-26T21:25:07.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:07 smithi171 ceph-mon[33478]: pgmap v564: 97 pgs: 97 active+clean; 734 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 1.01k op/s 2024-04-26T21:25:09.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:09 smithi150 ceph-mon[32290]: pgmap v565: 97 pgs: 97 active+clean; 723 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 956 op/s 2024-04-26T21:25:09.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:09 smithi129 ceph-mon[29237]: pgmap v565: 97 pgs: 97 active+clean; 723 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 956 op/s 2024-04-26T21:25:09.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:09 smithi171 ceph-mon[33478]: pgmap v565: 97 pgs: 97 active+clean; 723 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 956 op/s 2024-04-26T21:25:11.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:25:11.505 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.610+0000 7f22d3ff7640 1 -- 172.21.15.129:0/3637471278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 msgr2=0x7f22cc101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.610+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/3637471278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 0x7f22cc101c90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009900 tx=0x7f22c802a410 comp rx=0 tx=0).stop 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.610+0000 7f22d3ff7640 1 -- 172.21.15.129:0/3637471278 shutdown_connections 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.610+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/3637471278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22cc102a60 0x7f22cc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.610+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/3637471278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22cc103420 0x7f22cc109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.610+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/3637471278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 0x7f22cc101c90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.610+0000 7f22d3ff7640 1 -- 172.21.15.129:0/3637471278 >> 172.21.15.129:0/3637471278 conn(0x7f22cc0fcde0 msgr2=0x7f22cc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 -- 172.21.15.129:0/3637471278 shutdown_connections 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 -- 172.21.15.129:0/3637471278 wait complete. 2024-04-26T21:25:11.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 Processor -- start 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 -- start start 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 0x7f22cc105800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22cc102a60 0x7f22cc103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22cc103420 0x7f22cc104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc10d370 con 0x7f22cc101870 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc10d1f0 con 0x7f22cc103420 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d3ff7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc10d4f0 con 0x7f22cc102a60 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d1d93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 0x7f22cc105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d1d93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 0x7f22cc105800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50060/0 (socket says 172.21.15.129:50060) 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d1d93640 1 -- 172.21.15.129:0/4077393113 learned_addr learned my addr 172.21.15.129:0/4077393113 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d2594640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22cc103420 0x7f22cc104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.611+0000 7f22d1592640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22cc102a60 0x7f22cc103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22d1d93640 1 -- 172.21.15.129:0/4077393113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22cc102a60 msgr2=0x7f22cc103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22d1d93640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22cc102a60 0x7f22cc103e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22d1d93640 1 -- 172.21.15.129:0/4077393113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22cc103420 msgr2=0x7f22cc104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22d1d93640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22cc103420 0x7f22cc104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22d1d93640 1 -- 172.21.15.129:0/4077393113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc104c10 con 0x7f22cc101870 2024-04-26T21:25:11.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22d2594640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22cc103420 0x7f22cc104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:11.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22d1d93640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 0x7f22cc105800 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f22c8006010 tx=0x7f22c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:11.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22c2ffd640 1 -- 172.21.15.129:0/4077393113 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c803f030 con 0x7f22cc101870 2024-04-26T21:25:11.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22d3ff7640 1 -- 172.21.15.129:0/4077393113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22cc1a2d10 con 0x7f22cc101870 2024-04-26T21:25:11.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22c2ffd640 1 -- 172.21.15.129:0/4077393113 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c80318f0 con 0x7f22cc101870 2024-04-26T21:25:11.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22c2ffd640 1 -- 172.21.15.129:0/4077393113 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8035070 con 0x7f22cc101870 2024-04-26T21:25:11.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.612+0000 7f22d3ff7640 1 -- 172.21.15.129:0/4077393113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22cc1a31a0 con 0x7f22cc101870 2024-04-26T21:25:11.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.614+0000 7f22c2ffd640 1 -- 172.21.15.129:0/4077393113 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f22c8042030 con 0x7f22cc101870 2024-04-26T21:25:11.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.614+0000 7f22d6def740 1 Processor -- start 2024-04-26T21:25:11.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d6def740 1 -- start start 2024-04-26T21:25:11.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d6def740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 0x55deee46f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d6def740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee470530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d6def740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee470a70 0x55deee477300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d6def740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55deee3d3d80 con 0x55deee4700b0 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d6def740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55deee38d8b0 con 0x55deee470a70 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d6def740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55deee3cfd40 con 0x55deee46eee0 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d1d93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee470530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22c2ffd640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f22b80a3c40 0x7f22b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d1592640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee470a70 0x55deee477300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d1d93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee470530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50064/0 (socket says 172.21.15.129:50064) 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d1d93640 1 -- 172.21.15.129:0/4186669633 learned_addr learned my addr 172.21.15.129:0/4186669633 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d2594640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 0x55deee46f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d2594640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f22b80a3c40 0x7f22b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d1592640 1 -- 172.21.15.129:0/4186669633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 msgr2=0x55deee46f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:11.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d1592640 1 --2- 172.21.15.129:0/4186669633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 0x55deee46f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d1592640 1 -- 172.21.15.129:0/4186669633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 msgr2=0x55deee470530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d1592640 1 --2- 172.21.15.129:0/4186669633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee470530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22c2ffd640 1 -- 172.21.15.129:0/4077393113 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f22c80f0a50 con 0x7f22cc101870 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.615+0000 7f22d1592640 1 -- 172.21.15.129:0/4186669633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deee35ea40 con 0x55deee470a70 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.616+0000 7f22d1592640 1 --2- 172.21.15.129:0/4186669633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee470a70 0x55deee477300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009bf0 tx=0x7f22bc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af33700c77378a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.616+0000 7f22d2594640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f22b80a3c40 0x7f22b80a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55deee46ffb0 tx=0x7f22c4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.616+0000 7f22b7fff640 1 -- 172.21.15.129:0/4186669633 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc009da0 con 0x55deee470a70 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.616+0000 7f22b7fff640 1 -- 172.21.15.129:0/4186669633 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22bc005020 con 0x55deee470a70 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.616+0000 7f22b7fff640 1 -- 172.21.15.129:0/4186669633 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc004b20 con 0x55deee470a70 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 -- 172.21.15.129:0/4186669633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee470a70 msgr2=0x55deee477300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 --2- 172.21.15.129:0/4186669633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee470a70 0x55deee477300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009bf0 tx=0x7f22bc011810 comp rx=0 tx=0).stop 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 -- 172.21.15.129:0/4186669633 shutdown_connections 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 --2- 172.21.15.129:0/4186669633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee470a70 0x55deee477300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 --2- 172.21.15.129:0/4186669633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee470530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 --2- 172.21.15.129:0/4186669633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 0x55deee46f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 -- 172.21.15.129:0/4186669633 >> 172.21.15.129:0/4186669633 conn(0x55deee46a3d0 msgr2=0x55deee46a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 -- 172.21.15.129:0/4186669633 shutdown_connections 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 -- 172.21.15.129:0/4186669633 wait complete. 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 -- start start 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.617+0000 7f22d6def740 1 Processor -- start 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d6def740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee46eee0 0x55deee479360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d6def740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4798a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d6def740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 0x55deee4808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d6def740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55deee477a80 con 0x55deee46eee0 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d6def740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55deee35ad10 con 0x55deee4700b0 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d6def740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55deee477c00 con 0x55deee470a70 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d6def740 10 client.0 _ll_register_callbacks cb 0x55deee387210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:11.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1d93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee46eee0 0x55deee479360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1592640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4798a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1d93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee46eee0 0x55deee479360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50076/0 (socket says 172.21.15.129:50076) 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1d93640 1 -- 172.21.15.129:0/1455756116 learned_addr learned my addr 172.21.15.129:0/1455756116 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d2594640 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 0x55deee4808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1592640 1 -- 172.21.15.129:0/1455756116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 msgr2=0x55deee4808e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1592640 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 0x55deee4808e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1592640 1 -- 172.21.15.129:0/1455756116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee46eee0 msgr2=0x55deee479360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1592640 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee46eee0 0x55deee479360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1592640 1 -- 172.21.15.129:0/1455756116 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deee1af7d0 con 0x55deee4700b0 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1d93640 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee46eee0 0x55deee479360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d1592640 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4798a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0057c0 tx=0x7f22bc009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c25d3c4064cb825b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22b57fa640 1 -- 172.21.15.129:0/1455756116 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc029020 con 0x55deee4700b0 2024-04-26T21:25:11.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22d6def740 1 -- 172.21.15.129:0/1455756116 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22bc005020 con 0x55deee4700b0 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.618+0000 7f22b57fa640 1 -- 172.21.15.129:0/1455756116 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f22bc004ab0 con 0x55deee4700b0 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22b57fa640 1 -- 172.21.15.129:0/1455756116 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f22bc034050 con 0x55deee4700b0 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22d6def740 10 client.25589 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22d6def740 1 -- 172.21.15.129:0/1455756116 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55deee0e53f0 con 0x55deee4700b0 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22b57fa640 1 -- 172.21.15.129:0/1455756116 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22bc005020 con 0x55deee4700b0 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22b57fa640 1 -- 172.21.15.129:0/1455756116 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f22bc023420 con 0x55deee4700b0 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22d6def740 10 client.25589 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22d6def740 10 client.25589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22d6def740 20 client.25589 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22d6def740 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55deee484be0 0x55deee487030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22d6def740 4 client.25589 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22d6def740 1 -- 172.21.15.129:0/1455756116 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deee3e1990 con 0x55deee484be0 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.619+0000 7f22d1d93640 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55deee484be0 0x55deee487030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.620+0000 7f22d1d93640 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55deee484be0 0x55deee487030 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a317974a373f4ddf server_cookie=9cc9539fd7ff789e in_seq=0 out_seq=0 2024-04-26T21:25:11.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.620+0000 7f22b57fa640 10 client.25589 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:11.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.621+0000 7f22b57fa640 1 -- 172.21.15.129:0/1455756116 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55deee3e1990 con 0x55deee484be0 2024-04-26T21:25:11.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.621+0000 7f22b57fa640 10 client.25589 handle_command_reply: tid=0 2024-04-26T21:25:11.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.621+0000 7f22d6def740 1 client.25589 shutdown 2024-04-26T21:25:11.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.621+0000 7f22d6def740 1 -- 172.21.15.129:0/1455756116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 msgr2=0x55deee4798a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:11.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.621+0000 7f22d6def740 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4798a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0057c0 tx=0x7f22bc009fd0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22d6def740 1 -- 172.21.15.129:0/1455756116 shutdown_connections 2024-04-26T21:25:11.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22d6def740 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55deee484be0 0x55deee487030 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22d6def740 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4798a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22d6def740 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 0x55deee4808e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22b57fa640 0 client.25589 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:11.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22d6def740 1 --2- 172.21.15.129:0/1455756116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee46eee0 0x55deee479360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22d6def740 1 -- 172.21.15.129:0/1455756116 >> 172.21.15.129:0/1455756116 conn(0x55deee46a3d0 msgr2=0x55deee469550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:11.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22d6def740 1 -- 172.21.15.129:0/1455756116 shutdown_connections 2024-04-26T21:25:11.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22d6def740 1 -- 172.21.15.129:0/1455756116 wait complete. 2024-04-26T21:25:11.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.622+0000 7f22d6def740 20 client.25589 trim_cache size 0 max 16384 2024-04-26T21:25:11.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.639+0000 7f22d6def740 1 Processor -- start 2024-04-26T21:25:11.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d6def740 1 -- start start 2024-04-26T21:25:11.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d6def740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 0x55deee465480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d6def740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d6def740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee470a70 0x55deee46d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d6def740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1aa010 con 0x55deee470a70 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d6def740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1ac890 con 0x55deee4700b0 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d6def740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1aa680 con 0x55deee46eee0 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d2594640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee470a70 0x55deee46d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d2594640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee470a70 0x55deee46d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50090/0 (socket says 172.21.15.129:50090) 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d2594640 1 -- 172.21.15.129:0/3722642152 learned_addr learned my addr 172.21.15.129:0/3722642152 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d1592640 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d1d93640 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 0x55deee465480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d1592640 1 -- 172.21.15.129:0/3722642152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 msgr2=0x55deee465480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d1592640 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 0x55deee465480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d1592640 1 -- 172.21.15.129:0/3722642152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee470a70 msgr2=0x55deee46d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d1592640 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee470a70 0x55deee46d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d1592640 1 -- 172.21.15.129:0/3722642152 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deee0a50f0 con 0x55deee4700b0 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d1d93640 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 0x55deee465480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d2594640 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee470a70 0x55deee46d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22d1592640 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4808a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22bc023810 tx=0x7f22bc005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e56a5a79bd34886 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.640+0000 7f22b77fe640 1 -- 172.21.15.129:0/3722642152 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc029030 con 0x55deee4700b0 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22b77fe640 1 -- 172.21.15.129:0/3722642152 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22bc016070 con 0x55deee4700b0 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22b77fe640 1 -- 172.21.15.129:0/3722642152 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0042d0 con 0x55deee4700b0 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 -- 172.21.15.129:0/3722642152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 msgr2=0x55deee4808a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4808a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22bc023810 tx=0x7f22bc005620 comp rx=0 tx=0).stop 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 -- 172.21.15.129:0/3722642152 shutdown_connections 2024-04-26T21:25:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee4700b0 0x55deee4808a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee470a70 0x55deee46d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 --2- 172.21.15.129:0/3722642152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee46eee0 0x55deee465480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 -- 172.21.15.129:0/3722642152 >> 172.21.15.129:0/3722642152 conn(0x55deee46a3d0 msgr2=0x55deee46a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 -- 172.21.15.129:0/3722642152 shutdown_connections 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 -- 172.21.15.129:0/3722642152 wait complete. 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.641+0000 7f22d6def740 1 -- start start 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d6def740 1 Processor -- start 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d6def740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee46eee0 0x55deee47d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d6def740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee47dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d6def740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 0x55deee4e9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d6def740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1aad50 con 0x55deee4700b0 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d6def740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1ac1c0 con 0x55deee46eee0 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d6def740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1ab420 con 0x55deee470a70 2024-04-26T21:25:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d6def740 10 client.0 _ll_register_callbacks cb 0x7f22c80bca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1592640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee47dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1d93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee46eee0 0x55deee47d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1592640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee47dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50092/0 (socket says 172.21.15.129:50092) 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1592640 1 -- 172.21.15.129:0/788369844 learned_addr learned my addr 172.21.15.129:0/788369844 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d2594640 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 0x55deee4e9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1592640 1 -- 172.21.15.129:0/788369844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 msgr2=0x55deee4e9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1592640 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 0x55deee4e9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1592640 1 -- 172.21.15.129:0/788369844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee46eee0 msgr2=0x55deee47d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1592640 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee46eee0 0x55deee47d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1592640 1 -- 172.21.15.129:0/788369844 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deee3944b0 con 0x55deee4700b0 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1d93640 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee46eee0 0x55deee47d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22d1592640 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee47dde0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f22bc023ac0 tx=0x7f22bc0096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b27252c418fbf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.642+0000 7f22b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22b57fa640 1 -- 172.21.15.129:0/788369844 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc029040 con 0x55deee4700b0 2024-04-26T21:25:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22b57fa640 1 -- 172.21.15.129:0/788369844 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f22bc004cb0 con 0x55deee4700b0 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d6def740 1 -- 172.21.15.129:0/788369844 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22bc016070 con 0x55deee4700b0 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22b57fa640 1 -- 172.21.15.129:0/788369844 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f22bc0097a0 con 0x55deee4700b0 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d6def740 10 client.15999 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d6def740 1 -- 172.21.15.129:0/788369844 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55deee4ea7f0 con 0x55deee4700b0 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22b57fa640 1 -- 172.21.15.129:0/788369844 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22bc016070 con 0x55deee4700b0 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22b57fa640 1 -- 172.21.15.129:0/788369844 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f22bc0035f0 con 0x55deee4700b0 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d6def740 10 client.15999 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d6def740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d6def740 20 client.15999 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d6def740 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55deee4eb0c0 0x55deee4ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d6def740 4 client.15999 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:25:11.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d6def740 1 -- 172.21.15.129:0/788369844 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55deee3db800 con 0x55deee4eb0c0 2024-04-26T21:25:11.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.643+0000 7f22d1d93640 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55deee4eb0c0 0x55deee4ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:11.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.644+0000 7f22b57fa640 10 client.15999 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:11.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.644+0000 7f22d1d93640 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55deee4eb0c0 0x55deee4ed4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4665ab466041072 server_cookie=fb3239d95022d92c in_seq=0 out_seq=0 2024-04-26T21:25:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22b57fa640 1 -- 172.21.15.129:0/788369844 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55deee3db800 con 0x55deee4eb0c0 2024-04-26T21:25:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22b57fa640 10 client.15999 handle_command_reply: tid=0 2024-04-26T21:25:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 client.15999 shutdown 2024-04-26T21:25:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 -- 172.21.15.129:0/788369844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 msgr2=0x55deee47dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee47dde0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f22bc023ac0 tx=0x7f22bc0096b0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 -- 172.21.15.129:0/788369844 shutdown_connections 2024-04-26T21:25:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55deee4eb0c0 0x55deee4ed4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55deee470a70 0x55deee4e9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55deee4700b0 0x55deee47dde0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 --2- 172.21.15.129:0/788369844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55deee46eee0 0x55deee47d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22b57fa640 0 client.15999 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 -- 172.21.15.129:0/788369844 >> 172.21.15.129:0/788369844 conn(0x55deee46a3d0 msgr2=0x55deee465480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 -- 172.21.15.129:0/788369844 shutdown_connections 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.647+0000 7f22d6def740 1 -- 172.21.15.129:0/788369844 wait complete. 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d6def740 20 client.15999 trim_cache size 0 max 16384 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (10531 inodes in the stack)", 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: "952ef762-6180-47c4-829b-d71b70ddd250": { 2024-04-26T21:25:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:25:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "952ef762-6180-47c4-829b-d71b70ddd250", 2024-04-26T21:25:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:25:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:25:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:25:11.654 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 -- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f22b80a3c40 msgr2=0x7f22b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f22b80a3c40 0x7f22b80a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55deee46ffb0 tx=0x7f22c4004ab0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 -- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 msgr2=0x7f22cc105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 0x7f22cc105800 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f22c8006010 tx=0x7f22c80042f0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 -- 172.21.15.129:0/4077393113 shutdown_connections 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f22b80a3c40 0x7f22b80a6100 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22cc103420 0x7f22cc104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22cc102a60 0x7f22cc103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 --2- 172.21.15.129:0/4077393113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22cc101870 0x7f22cc105800 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 -- 172.21.15.129:0/4077393113 >> 172.21.15.129:0/4077393113 conn(0x7f22cc0fcde0 msgr2=0x7f22cc069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 -- 172.21.15.129:0/4077393113 shutdown_connections 2024-04-26T21:25:11.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:11.648+0000 7f22d3ff7640 1 -- 172.21.15.129:0/4077393113 wait complete. 2024-04-26T21:25:11.661 INFO:tasks.cephfs.filesystem:scrub status for tag:952ef762-6180-47c4-829b-d71b70ddd250 - {'path': '/', 'tag': '952ef762-6180-47c4-829b-d71b70ddd250', 'options': 'recursive,force'} 2024-04-26T21:25:11.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:11 smithi150 ceph-mon[32290]: pgmap v566: 97 pgs: 97 active+clean; 712 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 973 op/s 2024-04-26T21:25:11.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:11 smithi129 ceph-mon[29237]: pgmap v566: 97 pgs: 97 active+clean; 712 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 973 op/s 2024-04-26T21:25:11.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:11 smithi171 ceph-mon[33478]: pgmap v566: 97 pgs: 97 active+clean; 712 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 973 op/s 2024-04-26T21:25:13.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:13 smithi150 ceph-mon[32290]: pgmap v567: 97 pgs: 97 active+clean; 708 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.01k op/s 2024-04-26T21:25:13.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:13 smithi129 ceph-mon[29237]: pgmap v567: 97 pgs: 97 active+clean; 708 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.01k op/s 2024-04-26T21:25:13.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:13 smithi171 ceph-mon[33478]: pgmap v567: 97 pgs: 97 active+clean; 708 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.01k op/s 2024-04-26T21:25:15.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:15 smithi171 ceph-mon[33478]: pgmap v568: 97 pgs: 97 active+clean; 703 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 898 op/s 2024-04-26T21:25:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:15 smithi150 ceph-mon[32290]: pgmap v568: 97 pgs: 97 active+clean; 703 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 898 op/s 2024-04-26T21:25:15.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:15 smithi129 ceph-mon[29237]: pgmap v568: 97 pgs: 97 active+clean; 703 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 898 op/s 2024-04-26T21:25:17.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:17 smithi150 ceph-mon[32290]: pgmap v569: 97 pgs: 97 active+clean; 694 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.04k op/s 2024-04-26T21:25:17.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:17 smithi129 ceph-mon[29237]: pgmap v569: 97 pgs: 97 active+clean; 694 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.04k op/s 2024-04-26T21:25:17.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:17 smithi171 ceph-mon[33478]: pgmap v569: 97 pgs: 97 active+clean; 694 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.04k op/s 2024-04-26T21:25:19.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:19 smithi150 ceph-mon[32290]: pgmap v570: 97 pgs: 97 active+clean; 685 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.11k op/s 2024-04-26T21:25:19.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:19 smithi129 ceph-mon[29237]: pgmap v570: 97 pgs: 97 active+clean; 685 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.11k op/s 2024-04-26T21:25:19.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:19 smithi171 ceph-mon[33478]: pgmap v570: 97 pgs: 97 active+clean; 685 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.11k op/s 2024-04-26T21:25:21.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:21 smithi150 ceph-mon[32290]: pgmap v571: 97 pgs: 97 active+clean; 666 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.16k op/s 2024-04-26T21:25:21.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:21 smithi129 ceph-mon[29237]: pgmap v571: 97 pgs: 97 active+clean; 666 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.16k op/s 2024-04-26T21:25:21.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:21 smithi171 ceph-mon[33478]: pgmap v571: 97 pgs: 97 active+clean; 666 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.16k op/s 2024-04-26T21:25:23.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:23 smithi150 ceph-mon[32290]: pgmap v572: 97 pgs: 97 active+clean; 637 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 1.18k op/s 2024-04-26T21:25:23.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:23 smithi129 ceph-mon[29237]: pgmap v572: 97 pgs: 97 active+clean; 637 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 1.18k op/s 2024-04-26T21:25:23.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:23 smithi171 ceph-mon[33478]: pgmap v572: 97 pgs: 97 active+clean; 637 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 1.18k op/s 2024-04-26T21:25:24.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:24 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:25:24.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:24 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:25:24.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:24 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:25:25.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:25 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:25:25.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:25 smithi150 ceph-mon[32290]: pgmap v573: 97 pgs: 97 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 1.02k op/s 2024-04-26T21:25:25.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:25 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:25:25.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:25 smithi129 ceph-mon[29237]: pgmap v573: 97 pgs: 97 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 1.02k op/s 2024-04-26T21:25:25.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:25 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:25:25.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:25 smithi171 ceph-mon[33478]: pgmap v573: 97 pgs: 97 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 1.02k op/s 2024-04-26T21:25:27.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:27 smithi150 ceph-mon[32290]: pgmap v574: 97 pgs: 97 active+clean; 617 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 1.09k op/s 2024-04-26T21:25:27.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:27 smithi129 ceph-mon[29237]: pgmap v574: 97 pgs: 97 active+clean; 617 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 1.09k op/s 2024-04-26T21:25:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:27 smithi171 ceph-mon[33478]: pgmap v574: 97 pgs: 97 active+clean; 617 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 1.09k op/s 2024-04-26T21:25:29.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:29 smithi150 ceph-mon[32290]: pgmap v575: 97 pgs: 97 active+clean; 610 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 966 op/s 2024-04-26T21:25:29.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:29 smithi129 ceph-mon[29237]: pgmap v575: 97 pgs: 97 active+clean; 610 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 966 op/s 2024-04-26T21:25:29.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:29 smithi171 ceph-mon[33478]: pgmap v575: 97 pgs: 97 active+clean; 610 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 966 op/s 2024-04-26T21:25:31.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:31 smithi150 ceph-mon[32290]: pgmap v576: 97 pgs: 97 active+clean; 596 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 821 op/s 2024-04-26T21:25:31.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:31 smithi129 ceph-mon[29237]: pgmap v576: 97 pgs: 97 active+clean; 596 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 821 op/s 2024-04-26T21:25:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:31 smithi171 ceph-mon[33478]: pgmap v576: 97 pgs: 97 active+clean; 596 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 821 op/s 2024-04-26T21:25:33.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:33 smithi150 ceph-mon[32290]: pgmap v577: 97 pgs: 97 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 722 op/s 2024-04-26T21:25:33.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:33 smithi129 ceph-mon[29237]: pgmap v577: 97 pgs: 97 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 722 op/s 2024-04-26T21:25:33.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:33 smithi171 ceph-mon[33478]: pgmap v577: 97 pgs: 97 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 722 op/s 2024-04-26T21:25:35.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:35 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:35.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:35 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:35.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:35 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:36.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:36 smithi150 ceph-mon[32290]: pgmap v578: 97 pgs: 97 active+clean; 573 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 549 op/s 2024-04-26T21:25:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:36 smithi129 ceph-mon[29237]: pgmap v578: 97 pgs: 97 active+clean; 573 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 549 op/s 2024-04-26T21:25:36.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:36 smithi171 ceph-mon[33478]: pgmap v578: 97 pgs: 97 active+clean; 573 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 549 op/s 2024-04-26T21:25:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:36.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:37 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-26T21:25:37.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:37 smithi129 ceph-mon[29237]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-26T21:25:37.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.621 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:37.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:37.622 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:25:37.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:37 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-26T21:25:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:37 smithi150 ceph-mon[32290]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-26T21:25:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:25:37.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:37 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-26T21:25:37.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:37 smithi171 ceph-mon[33478]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-26T21:25:37.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:37.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:37.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:37.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:25:38.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:38 smithi150 ceph-mon[32290]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-26T21:25:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:38 smithi150 ceph-mon[32290]: Unable to set osd_memory_target on smithi150 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-26T21:25:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:38 smithi150 ceph-mon[32290]: pgmap v579: 97 pgs: 97 active+clean; 554 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 555 op/s 2024-04-26T21:25:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:38 smithi129 ceph-mon[29237]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-26T21:25:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:38 smithi129 ceph-mon[29237]: Unable to set osd_memory_target on smithi150 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-26T21:25:38.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:38 smithi129 ceph-mon[29237]: pgmap v579: 97 pgs: 97 active+clean; 554 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 555 op/s 2024-04-26T21:25:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:38 smithi171 ceph-mon[33478]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-26T21:25:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:38 smithi171 ceph-mon[33478]: Unable to set osd_memory_target on smithi150 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-26T21:25:38.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:38 smithi171 ceph-mon[33478]: pgmap v579: 97 pgs: 97 active+clean; 554 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 555 op/s 2024-04-26T21:25:39.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:39 smithi150 ceph-mon[32290]: pgmap v580: 97 pgs: 97 active+clean; 531 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 536 op/s 2024-04-26T21:25:39.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:39 smithi129 ceph-mon[29237]: pgmap v580: 97 pgs: 97 active+clean; 531 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 536 op/s 2024-04-26T21:25:39.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:39 smithi171 ceph-mon[33478]: pgmap v580: 97 pgs: 97 active+clean; 531 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 536 op/s 2024-04-26T21:25:41.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:25:41.662 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:25:41.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.765+0000 7f599e7a4640 1 -- 172.21.15.129:0/520199515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5998105d20 msgr2=0x7f5998108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.765+0000 7f599e7a4640 1 --2- 172.21.15.129:0/520199515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5998105d20 0x7f5998108110 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5988009900 tx=0x7f598802a410 comp rx=0 tx=0).stop 2024-04-26T21:25:41.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.765+0000 7f599e7a4640 1 -- 172.21.15.129:0/520199515 shutdown_connections 2024-04-26T21:25:41.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.765+0000 7f599e7a4640 1 --2- 172.21.15.129:0/520199515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5998108650 0x7f599810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.765+0000 7f599e7a4640 1 --2- 172.21.15.129:0/520199515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5998105d20 0x7f5998108110 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.765+0000 7f599e7a4640 1 --2- 172.21.15.129:0/520199515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59981033f0 0x7f59981057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.765+0000 7f599e7a4640 1 -- 172.21.15.129:0/520199515 >> 172.21.15.129:0/520199515 conn(0x7f59980fcde0 msgr2=0x7f59980ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:41.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.765+0000 7f599e7a4640 1 -- 172.21.15.129:0/520199515 shutdown_connections 2024-04-26T21:25:41.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.765+0000 7f599e7a4640 1 -- 172.21.15.129:0/520199515 wait complete. 2024-04-26T21:25:41.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f599e7a4640 1 Processor -- start 2024-04-26T21:25:41.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f599e7a4640 1 -- start start 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f599e7a4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59981033f0 0x7f5998102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f599e7a4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5998105d20 0x7f59981013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f599e7a4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5998108650 0x7f59981018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f599e7a4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f599810e490 con 0x7f59981033f0 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f599e7a4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f599810e310 con 0x7f5998108650 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f599e7a4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f599810e610 con 0x7f5998105d20 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f5997fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59981033f0 0x7f5998102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f5997fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59981033f0 0x7f5998102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43456/0 (socket says 172.21.15.129:43456) 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f5997fff640 1 -- 172.21.15.129:0/3920481539 learned_addr learned my addr 172.21.15.129:0/3920481539 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:41.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.766+0000 7f599cd41640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5998108650 0x7f59981018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f59977fe640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5998105d20 0x7f59981013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f5997fff640 1 -- 172.21.15.129:0/3920481539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5998105d20 msgr2=0x7f59981013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f5997fff640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5998105d20 0x7f59981013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f5997fff640 1 -- 172.21.15.129:0/3920481539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5998108650 msgr2=0x7f59981018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f5997fff640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5998108650 0x7f59981018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f5997fff640 1 -- 172.21.15.129:0/3920481539 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5998102170 con 0x7f59981033f0 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f5997fff640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59981033f0 0x7f5998102d60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f598c00e9f0 tx=0x7f598c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f59957fa640 1 -- 172.21.15.129:0/3920481539 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c00cd80 con 0x7f59981033f0 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f599e7a4640 1 -- 172.21.15.129:0/3920481539 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59981a2cb0 con 0x7f59981033f0 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f59957fa640 1 -- 172.21.15.129:0/3920481539 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f598c0108e0 con 0x7f59981033f0 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f59957fa640 1 -- 172.21.15.129:0/3920481539 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c018980 con 0x7f59981033f0 2024-04-26T21:25:41.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.767+0000 7f599e7a4640 1 -- 172.21.15.129:0/3920481539 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59981a31d0 con 0x7f59981033f0 2024-04-26T21:25:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.769+0000 7f59957fa640 1 -- 172.21.15.129:0/3920481539 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f598c010430 con 0x7f59981033f0 2024-04-26T21:25:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59a1597740 1 Processor -- start 2024-04-26T21:25:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59a1597740 1 -- start start 2024-04-26T21:25:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59a1597740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed6182300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59a1597740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 0x559ed6183530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59a1597740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 0x559ed618a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59957fa640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59800a3c40 0x7f59800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59a1597740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ed60e6d80 con 0x559ed6181ee0 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59a1597740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559ed60a08b0 con 0x559ed61830b0 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59a1597740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559ed60e2d40 con 0x559ed6183a70 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f599cd41640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed6182300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f5997fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 0x559ed6183530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f599cd41640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed6182300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43460/0 (socket says 172.21.15.129:43460) 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59977fe640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59800a3c40 0x7f59800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f599cd41640 1 -- 172.21.15.129:0/831770373 learned_addr learned my addr 172.21.15.129:0/831770373 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f59957fa640 1 -- 172.21.15.129:0/3920481539 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f598c0d0890 con 0x7f59981033f0 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f599cd41640 1 --2- 172.21.15.129:0/831770373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 0x559ed618a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f5997fff640 1 -- 172.21.15.129:0/831770373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 msgr2=0x559ed618a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f5997fff640 1 --2- 172.21.15.129:0/831770373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 0x559ed618a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f5997fff640 1 -- 172.21.15.129:0/831770373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 msgr2=0x559ed6182300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f5997fff640 1 --2- 172.21.15.129:0/831770373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed6182300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f5997fff640 1 -- 172.21.15.129:0/831770373 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed6071a40 con 0x559ed61830b0 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.770+0000 7f599cd41640 1 --2- 172.21.15.129:0/831770373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed6182300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.771+0000 7f59977fe640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59800a3c40 0x7f59800a6100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5988009230 tx=0x7f59880314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.771+0000 7f5997fff640 1 --2- 172.21.15.129:0/831770373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 0x559ed6183530 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f598c0ce230 tx=0x7f598c06e600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab502f5c8d6cc327 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.771+0000 7f597e7fc640 1 -- 172.21.15.129:0/831770373 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c0a4370 con 0x559ed61830b0 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.771+0000 7f597e7fc640 1 -- 172.21.15.129:0/831770373 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f598c010720 con 0x559ed61830b0 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.771+0000 7f597e7fc640 1 -- 172.21.15.129:0/831770373 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c0a47a0 con 0x559ed61830b0 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 -- 172.21.15.129:0/831770373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 msgr2=0x559ed6183530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 --2- 172.21.15.129:0/831770373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 0x559ed6183530 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f598c0ce230 tx=0x7f598c06e600 comp rx=0 tx=0).stop 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 -- 172.21.15.129:0/831770373 shutdown_connections 2024-04-26T21:25:41.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 --2- 172.21.15.129:0/831770373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 0x559ed6183530 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 --2- 172.21.15.129:0/831770373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 0x559ed618a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 --2- 172.21.15.129:0/831770373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed6182300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 -- 172.21.15.129:0/831770373 >> 172.21.15.129:0/831770373 conn(0x559ed617d3d0 msgr2=0x559ed617d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 -- 172.21.15.129:0/831770373 shutdown_connections 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 -- 172.21.15.129:0/831770373 wait complete. 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 -- start start 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 Processor -- start 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed61830b0 0x559ed618c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6183a70 0x559ed6193940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ed618aa80 con 0x559ed6183a70 2024-04-26T21:25:41.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559ed606dd10 con 0x559ed6181ee0 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559ed618ac00 con 0x559ed61830b0 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f59a1597740 10 client.0 _ll_register_callbacks cb 0x559ed609a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f599cd41640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6183a70 0x559ed6193940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f599cd41640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6183a70 0x559ed6193940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43474/0 (socket says 172.21.15.129:43474) 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.772+0000 7f599cd41640 1 -- 172.21.15.129:0/224355043 learned_addr learned my addr 172.21.15.129:0/224355043 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f5997fff640 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f59977fe640 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed61830b0 0x559ed618c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f599cd41640 1 -- 172.21.15.129:0/224355043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed61830b0 msgr2=0x559ed618c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f599cd41640 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed61830b0 0x559ed618c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f599cd41640 1 -- 172.21.15.129:0/224355043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6181ee0 msgr2=0x559ed618c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:41.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f599cd41640 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f599cd41640 1 -- 172.21.15.129:0/224355043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed5ec27d0 con 0x559ed6183a70 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f5997fff640 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f599cd41640 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6183a70 0x559ed6193940 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f59900042a0 tx=0x7f5990004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a12f60add636ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f59777fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f59777fe640 1 -- 172.21.15.129:0/224355043 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990019070 con 0x559ed6183a70 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f59777fe640 1 -- 172.21.15.129:0/224355043 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f599001e460 con 0x559ed6183a70 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f59a1597740 1 -- 172.21.15.129:0/224355043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559ed60f4990 con 0x559ed6183a70 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f59777fe640 1 -- 172.21.15.129:0/224355043 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f599001e660 con 0x559ed6183a70 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f59a1597740 10 client.16011 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f59a1597740 1 -- 172.21.15.129:0/224355043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559ed5df83f0 con 0x559ed6183a70 2024-04-26T21:25:41.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.773+0000 7f59777fe640 1 -- 172.21.15.129:0/224355043 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559ed60f4990 con 0x559ed6183a70 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f59777fe640 1 -- 172.21.15.129:0/224355043 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f5990028370 con 0x559ed6183a70 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f59a1597740 10 client.16011 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f59a1597740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f59a1597740 20 client.16011 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f59a1597740 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x559ed6197f20 0x559ed619a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f59a1597740 4 client.16011 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f59a1597740 1 -- 172.21.15.129:0/224355043 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ed60ee800 con 0x559ed6197f20 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f5997fff640 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x559ed6197f20 0x559ed619a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f59777fe640 10 client.16011 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.774+0000 7f5997fff640 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x559ed6197f20 0x559ed619a3a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec2e74ec0b6c8a4 server_cookie=68ac0270c6f9f9a1 in_seq=0 out_seq=0 2024-04-26T21:25:41.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59777fe640 1 -- 172.21.15.129:0/224355043 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ed60ee800 con 0x559ed6197f20 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59777fe640 10 client.16011 handle_command_reply: tid=0 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 client.16011 shutdown 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 -- 172.21.15.129:0/224355043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6183a70 msgr2=0x559ed6193940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6183a70 0x559ed6193940 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f59900042a0 tx=0x7f5990004760 comp rx=0 tx=0).stop 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 -- 172.21.15.129:0/224355043 shutdown_connections 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x559ed6197f20 0x559ed619a3a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6183a70 0x559ed6193940 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed61830b0 0x559ed618c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 --2- 172.21.15.129:0/224355043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59777fe640 0 client.16011 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 -- 172.21.15.129:0/224355043 >> 172.21.15.129:0/224355043 conn(0x559ed617d3d0 msgr2=0x559ed617c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 -- 172.21.15.129:0/224355043 shutdown_connections 2024-04-26T21:25:41.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 1 -- 172.21.15.129:0/224355043 wait complete. 2024-04-26T21:25:41.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.775+0000 7f59a1597740 20 client.16011 trim_cache size 0 max 16384 2024-04-26T21:25:41.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.793+0000 7f59a1597740 1 Processor -- start 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.793+0000 7f59a1597740 1 -- start start 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.793+0000 7f59a1597740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.793+0000 7f59a1597740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 0x559ed6193900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.793+0000 7f59a1597740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 0x559ed618c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.793+0000 7f59a1597740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f59981aa010 con 0x559ed6181ee0 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.793+0000 7f59a1597740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ac890 con 0x559ed61830b0 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.793+0000 7f59a1597740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f59981aa680 con 0x559ed6183a70 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f5997fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f5997fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43484/0 (socket says 172.21.15.129:43484) 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f5997fff640 1 -- 172.21.15.129:0/2710113652 learned_addr learned my addr 172.21.15.129:0/2710113652 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f59977fe640 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 0x559ed6193900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f599cd41640 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 0x559ed618c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f5997fff640 1 -- 172.21.15.129:0/2710113652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 msgr2=0x559ed618c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f5997fff640 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 0x559ed618c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f5997fff640 1 -- 172.21.15.129:0/2710113652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 msgr2=0x559ed6193900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f5997fff640 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 0x559ed6193900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f5997fff640 1 -- 172.21.15.129:0/2710113652 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed5db80f0 con 0x559ed6181ee0 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f599cd41640 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 0x559ed618c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f5997fff640 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f598c00e9c0 tx=0x7f598c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9973f7e1336074a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f597dffb640 1 -- 172.21.15.129:0/2710113652 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c09adb0 con 0x559ed6181ee0 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f597dffb640 1 -- 172.21.15.129:0/2710113652 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f598c0ce480 con 0x559ed6181ee0 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.794+0000 7f597dffb640 1 -- 172.21.15.129:0/2710113652 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c0bbbc0 con 0x559ed6181ee0 2024-04-26T21:25:41.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 -- 172.21.15.129:0/2710113652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 msgr2=0x559ed618c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f598c00e9c0 tx=0x7f598c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 -- 172.21.15.129:0/2710113652 shutdown_connections 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed61830b0 0x559ed6193900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6183a70 0x559ed618c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 --2- 172.21.15.129:0/2710113652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed6181ee0 0x559ed618c3c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 -- 172.21.15.129:0/2710113652 >> 172.21.15.129:0/2710113652 conn(0x559ed617d3d0 msgr2=0x559ed6178480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 -- 172.21.15.129:0/2710113652 shutdown_connections 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 -- 172.21.15.129:0/2710113652 wait complete. 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 -- start start 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 Processor -- start 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6181ee0 0x559ed6190850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed61830b0 0x559ed6190d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.795+0000 7f59a1597740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6183a70 0x559ed61fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59a1597740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f59981aad50 con 0x559ed61830b0 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59a1597740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ac1c0 con 0x559ed6183a70 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59a1597740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ab420 con 0x559ed6181ee0 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59a1597740 10 client.0 _ll_register_callbacks cb 0x7f59900071c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed61830b0 0x559ed6190d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed61830b0 0x559ed6190d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43486/0 (socket says 172.21.15.129:43486) 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59977fe640 1 -- 172.21.15.129:0/1440931945 learned_addr learned my addr 172.21.15.129:0/1440931945 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f599cd41640 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6183a70 0x559ed61fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59977fe640 1 -- 172.21.15.129:0/1440931945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6181ee0 msgr2=0x559ed6190850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59977fe640 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6181ee0 0x559ed6190850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59977fe640 1 -- 172.21.15.129:0/1440931945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6183a70 msgr2=0x559ed61fc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:41.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59977fe640 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6183a70 0x559ed61fc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59977fe640 1 -- 172.21.15.129:0/1440931945 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed60a74b0 con 0x559ed61830b0 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59977fe640 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed61830b0 0x559ed6190d90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5988009900 tx=0x7f598802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e769fa3c1fb2c8da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59777fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59777fe640 1 -- 172.21.15.129:0/1440931945 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988035050 con 0x559ed61830b0 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59777fe640 1 -- 172.21.15.129:0/1440931945 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5988003800 con 0x559ed61830b0 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59a1597740 1 -- 172.21.15.129:0/1440931945 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598c0ce480 con 0x559ed61830b0 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59777fe640 1 -- 172.21.15.129:0/1440931945 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5988003a00 con 0x559ed61830b0 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59a1597740 10 client.16017 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59a1597740 1 -- 172.21.15.129:0/1440931945 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559ed61fdb90 con 0x559ed61830b0 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.796+0000 7f59777fe640 1 -- 172.21.15.129:0/1440931945 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f598c0ce480 con 0x559ed61830b0 2024-04-26T21:25:41.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59777fe640 1 -- 172.21.15.129:0/1440931945 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f598805d050 con 0x559ed61830b0 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59a1597740 10 client.16017 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59a1597740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59a1597740 20 client.16017 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59a1597740 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x559ed61fe570 0x559ed6200990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59a1597740 4 client.16017 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59a1597740 1 -- 172.21.15.129:0/1440931945 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ed60694a0 con 0x559ed61fe570 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f5997fff640 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x559ed61fe570 0x559ed6200990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59777fe640 10 client.16017 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f5997fff640 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x559ed61fe570 0x559ed6200990 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86812a0979db58c2 server_cookie=3abf8c94804f5ffc in_seq=0 out_seq=0 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59777fe640 1 -- 172.21.15.129:0/1440931945 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559ed60694a0 con 0x559ed61fe570 2024-04-26T21:25:41.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59777fe640 10 client.16017 handle_command_reply: tid=0 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.797+0000 7f59a1597740 1 client.16017 shutdown 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 -- 172.21.15.129:0/1440931945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed61830b0 msgr2=0x559ed6190d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed61830b0 0x559ed6190d90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5988009900 tx=0x7f598802a410 comp rx=0 tx=0).stop 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 -- 172.21.15.129:0/1440931945 shutdown_connections 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x559ed61fe570 0x559ed6200990 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed6183a70 0x559ed61fc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59777fe640 0 client.16017 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ed61830b0 0x559ed6190d90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 --2- 172.21.15.129:0/1440931945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ed6181ee0 0x559ed6190850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 -- 172.21.15.129:0/1440931945 >> 172.21.15.129:0/1440931945 conn(0x559ed617d3d0 msgr2=0x559ed618c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 -- 172.21.15.129:0/1440931945 shutdown_connections 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 1 -- 172.21.15.129:0/1440931945 wait complete. 2024-04-26T21:25:41.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f59a1597740 20 client.16017 trim_cache size 0 max 16384 2024-04-26T21:25:41.808 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:25:41.808 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:25:41.808 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:25:41.808 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:25:41.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 -- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59800a3c40 msgr2=0x7f59800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59800a3c40 0x7f59800a6100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5988009230 tx=0x7f59880314a0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 -- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59981033f0 msgr2=0x7f5998102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59981033f0 0x7f5998102d60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f598c00e9f0 tx=0x7f598c00eeb0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 -- 172.21.15.129:0/3920481539 shutdown_connections 2024-04-26T21:25:41.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f59800a3c40 0x7f59800a6100 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5998108650 0x7f59981018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5998105d20 0x7f59981013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 --2- 172.21.15.129:0/3920481539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f59981033f0 0x7f5998102d60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 -- 172.21.15.129:0/3920481539 >> 172.21.15.129:0/3920481539 conn(0x7f59980fcde0 msgr2=0x7f5998103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:41.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 -- 172.21.15.129:0/3920481539 shutdown_connections 2024-04-26T21:25:41.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.798+0000 7f599e7a4640 1 -- 172.21.15.129:0/3920481539 wait complete. 2024-04-26T21:25:41.810 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:41 smithi129 ceph-mon[29237]: pgmap v581: 97 pgs: 97 active+clean; 513 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 525 op/s 2024-04-26T21:25:41.811 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:25:41.812 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:25:41.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:41 smithi150 ceph-mon[32290]: pgmap v581: 97 pgs: 97 active+clean; 513 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 525 op/s 2024-04-26T21:25:41.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.913+0000 7fa484df7640 1 -- 172.21.15.129:0/1890549233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4801086f0 msgr2=0x7fa48010aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.913+0000 7fa484df7640 1 --2- 172.21.15.129:0/1890549233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4801086f0 0x7fa48010aae0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa478009ca0 tx=0x7fa47802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.913+0000 7fa484df7640 1 -- 172.21.15.129:0/1890549233 shutdown_connections 2024-04-26T21:25:41.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.913+0000 7fa484df7640 1 --2- 172.21.15.129:0/1890549233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4801086f0 0x7fa48010aae0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.913+0000 7fa484df7640 1 --2- 172.21.15.129:0/1890549233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa480105dc0 0x7fa4801081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.913+0000 7fa484df7640 1 --2- 172.21.15.129:0/1890549233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa480103490 0x7fa480105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.913+0000 7fa484df7640 1 -- 172.21.15.129:0/1890549233 >> 172.21.15.129:0/1890549233 conn(0x7fa4800fcde0 msgr2=0x7fa4800ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:41.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.913+0000 7fa484df7640 1 -- 172.21.15.129:0/1890549233 shutdown_connections 2024-04-26T21:25:41.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.913+0000 7fa484df7640 1 -- 172.21.15.129:0/1890549233 wait complete. 2024-04-26T21:25:41.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa484df7640 1 Processor -- start 2024-04-26T21:25:41.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa484df7640 1 -- start start 2024-04-26T21:25:41.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa484df7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa480103490 0x7fa480198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa484df7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa480105dc0 0x7fa480199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa484df7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4801086f0 0x7fa4801a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa484df7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa48010e4a0 con 0x7fa480105dc0 2024-04-26T21:25:41.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa484df7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa48010e320 con 0x7fa4801086f0 2024-04-26T21:25:41.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa484df7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa48010e620 con 0x7fa480103490 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa475d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa480105dc0 0x7fa480199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa475d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa480105dc0 0x7fa480199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43512/0 (socket says 172.21.15.129:43512) 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa475d9b640 1 -- 172.21.15.129:0/1657350132 learned_addr learned my addr 172.21.15.129:0/1657350132 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa47ed9d640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4801086f0 0x7fa4801a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa47e59c640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa480103490 0x7fa480198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa475d9b640 1 -- 172.21.15.129:0/1657350132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa480103490 msgr2=0x7fa480198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa475d9b640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa480103490 0x7fa480198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa475d9b640 1 -- 172.21.15.129:0/1657350132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4801086f0 msgr2=0x7fa4801a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa475d9b640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4801086f0 0x7fa4801a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa475d9b640 1 -- 172.21.15.129:0/1657350132 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4801a0710 con 0x7fa480105dc0 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.914+0000 7fa47ed9d640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4801086f0 0x7fa4801a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.915+0000 7fa475d9b640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa480105dc0 0x7fa480199030 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa46800c970 tx=0x7fa46800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.915+0000 7fa477fff640 1 -- 172.21.15.129:0/1657350132 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468014070 con 0x7fa480105dc0 2024-04-26T21:25:41.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.915+0000 7fa477fff640 1 -- 172.21.15.129:0/1657350132 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa468004440 con 0x7fa480105dc0 2024-04-26T21:25:41.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.915+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4801a0a00 con 0x7fa480105dc0 2024-04-26T21:25:41.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.915+0000 7fa477fff640 1 -- 172.21.15.129:0/1657350132 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468003e50 con 0x7fa480105dc0 2024-04-26T21:25:41.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.915+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4801a0f50 con 0x7fa480105dc0 2024-04-26T21:25:41.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.916+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4800fe8d0 con 0x7fa480105dc0 2024-04-26T21:25:41.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.917+0000 7fa477fff640 1 -- 172.21.15.129:0/1657350132 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa468017070 con 0x7fa480105dc0 2024-04-26T21:25:41.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.917+0000 7fa477fff640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4600a3de0 0x7fa4600a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:41.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.918+0000 7fa47e59c640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4600a3de0 0x7fa4600a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:41.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.918+0000 7fa477fff640 1 -- 172.21.15.129:0/1657350132 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa4680d0830 con 0x7fa480105dc0 2024-04-26T21:25:41.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.918+0000 7fa47e59c640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4600a3de0 0x7fa4600a62a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa470009b60 tx=0x7fa470005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:41.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:41.921+0000 7fa477fff640 1 -- 172.21.15.129:0/1657350132 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa46801e030 con 0x7fa480105dc0 2024-04-26T21:25:41.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:41 smithi171 ceph-mon[33478]: pgmap v581: 97 pgs: 97 active+clean; 513 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 525 op/s 2024-04-26T21:25:42.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.149+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa48002dad0 con 0x7fa480105dc0 2024-04-26T21:25:42.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.150+0000 7fa477fff640 1 -- 172.21.15.129:0/1657350132 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7fa46809d0a0 con 0x7fa480105dc0 2024-04-26T21:25:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:25:42.152 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:25:42.152 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.153+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4600a3de0 msgr2=0x7fa4600a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.153+0000 7fa484df7640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4600a3de0 0x7fa4600a62a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa470009b60 tx=0x7fa470005f90 comp rx=0 tx=0).stop 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.153+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa480105dc0 msgr2=0x7fa480199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.153+0000 7fa484df7640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa480105dc0 0x7fa480199030 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa46800c970 tx=0x7fa46800ce30 comp rx=0 tx=0).stop 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.154+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 shutdown_connections 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.154+0000 7fa484df7640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa4600a3de0 0x7fa4600a62a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.154+0000 7fa484df7640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4801086f0 0x7fa4801a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.154+0000 7fa484df7640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa480105dc0 0x7fa480199030 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.154+0000 7fa484df7640 1 --2- 172.21.15.129:0/1657350132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa480103490 0x7fa480198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.154+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 >> 172.21.15.129:0/1657350132 conn(0x7fa4800fcde0 msgr2=0x7fa480103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:42.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.154+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 shutdown_connections 2024-04-26T21:25:42.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.154+0000 7fa484df7640 1 -- 172.21.15.129:0/1657350132 wait complete. 2024-04-26T21:25:42.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:25:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.305+0000 7fe59ffc6640 1 -- 172.21.15.129:0/3677909252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 msgr2=0x7fe59810adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.305+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/3677909252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 0x7fe59810adb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe590009ca0 tx=0x7fe59002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.305+0000 7fe59ffc6640 1 -- 172.21.15.129:0/3677909252 shutdown_connections 2024-04-26T21:25:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.305+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/3677909252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598103b60 0x7fe598103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.305+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/3677909252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 0x7fe59810adb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.305+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/3677909252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe598102970 0x7fe598102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.305+0000 7fe59ffc6640 1 -- 172.21.15.129:0/3677909252 >> 172.21.15.129:0/3677909252 conn(0x7fe5980fdee0 msgr2=0x7fe598100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 -- 172.21.15.129:0/3677909252 shutdown_connections 2024-04-26T21:25:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 -- 172.21.15.129:0/3677909252 wait complete. 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 Processor -- start 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 -- start start 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102970 0x7fe598198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe598103b60 0x7fe598199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 0x7fe5981a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe59810e4a0 con 0x7fe598104520 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe59810e320 con 0x7fe598102970 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59ffc6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe59810e620 con 0x7fe598103b60 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59e563640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 0x7fe5981a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59dd62640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102970 0x7fe598198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59dd62640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102970 0x7fe598198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:58988/0 (socket says 172.21.15.129:58988) 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59e563640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 0x7fe5981a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43530/0 (socket says 172.21.15.129:43530) 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59dd62640 1 -- 172.21.15.129:0/155298295 learned_addr learned my addr 172.21.15.129:0/155298295 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.306+0000 7fe59d561640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe598103b60 0x7fe598199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe59e563640 1 -- 172.21.15.129:0/155298295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe598103b60 msgr2=0x7fe598199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe59e563640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe598103b60 0x7fe598199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe59e563640 1 -- 172.21.15.129:0/155298295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102970 msgr2=0x7fe598198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe59e563640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102970 0x7fe598198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe59e563640 1 -- 172.21.15.129:0/155298295 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5981a07c0 con 0x7fe598104520 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe59dd62640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102970 0x7fe598198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe59e563640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 0x7fe5981a0150 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe590031a20 tx=0x7fe5900317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe58effd640 1 -- 172.21.15.129:0/155298295 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590003c30 con 0x7fe598104520 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe58effd640 1 -- 172.21.15.129:0/155298295 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe590003dd0 con 0x7fe598104520 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe59ffc6640 1 -- 172.21.15.129:0/155298295 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5981a0a50 con 0x7fe598104520 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe58effd640 1 -- 172.21.15.129:0/155298295 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59002fb60 con 0x7fe598104520 2024-04-26T21:25:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.307+0000 7fe59ffc6640 1 -- 172.21.15.129:0/155298295 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5981a0fa0 con 0x7fe598104520 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.309+0000 7fe58effd640 1 -- 172.21.15.129:0/155298295 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe59002fd20 con 0x7fe598104520 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.309+0000 7fe5a2dc0740 1 Processor -- start 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe5a2dc0740 1 -- start start 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6d9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 0x55617a6e1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55617a638d80 con 0x55617a6d8c30 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55617a5f2a20 con 0x55617a6da7c0 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55617a634d40 con 0x55617a6d9e00 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6d9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6d9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43546/0 (socket says 172.21.15.129:43546) 2024-04-26T21:25:42.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 -- 172.21.15.129:0/3062217243 learned_addr learned my addr 172.21.15.129:0/3062217243 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59dd62640 1 --2- 172.21.15.129:0/3062217243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe58effd640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe5840a3c40 0x7fe5840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 --2- 172.21.15.129:0/3062217243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 0x55617a6e1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59e563640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe5840a3c40 0x7fe5840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 -- 172.21.15.129:0/3062217243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 msgr2=0x55617a6da280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 --2- 172.21.15.129:0/3062217243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6da280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 -- 172.21.15.129:0/3062217243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 msgr2=0x55617a6e1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 --2- 172.21.15.129:0/3062217243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 0x55617a6e1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 -- 172.21.15.129:0/3062217243 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617a5fdc30 con 0x55617a6d8c30 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe58effd640 1 -- 172.21.15.129:0/155298295 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe5900f1990 con 0x7fe598104520 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.310+0000 7fe59d561640 1 --2- 172.21.15.129:0/3062217243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6d9050 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe594013620 tx=0x7fe594009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e77214d0492625 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe583fff640 1 -- 172.21.15.129:0/3062217243 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594004050 con 0x55617a6d8c30 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe59e563640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe5840a3c40 0x7fe5840a6100 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe5900bdc30 tx=0x7fe5900907e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:42.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe583fff640 1 -- 172.21.15.129:0/3062217243 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940041f0 con 0x55617a6d8c30 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe583fff640 1 -- 172.21.15.129:0/3062217243 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59400f7e0 con 0x55617a6d8c30 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3062217243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 msgr2=0x55617a6d9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/3062217243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6d9050 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe594013620 tx=0x7fe594009850 comp rx=0 tx=0).stop 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3062217243 shutdown_connections 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/3062217243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 0x55617a6e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/3062217243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6da280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/3062217243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6d9050 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3062217243 >> 172.21.15.129:0/3062217243 conn(0x55617a615910 msgr2=0x55617a6d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3062217243 shutdown_connections 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.311+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3062217243 wait complete. 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe5a2dc0740 1 -- start start 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe5a2dc0740 1 Processor -- start 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 0x55617a6ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55617a6e1770 con 0x55617a6d8c30 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55617a5bfdc0 con 0x55617a6da7c0 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55617a6e18f0 con 0x55617a6d9e00 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe5a2dc0740 10 client.0 _ll_register_callbacks cb 0x55617a5db600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe59d561640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe59d561640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6e2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43552/0 (socket says 172.21.15.129:43552) 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe59d561640 1 -- 172.21.15.129:0/941706383 learned_addr learned my addr 172.21.15.129:0/941706383 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe59dd62640 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.312+0000 7fe59d561640 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 0x55617a6ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe59d561640 1 -- 172.21.15.129:0/941706383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 msgr2=0x55617a6e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe59d561640 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6e3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe59d561640 1 -- 172.21.15.129:0/941706383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 msgr2=0x55617a6ea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe59d561640 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 0x55617a6ea3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe59d561640 1 -- 172.21.15.129:0/941706383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617a34a3f0 con 0x55617a6d8c30 2024-04-26T21:25:42.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe59d561640 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6e2e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe594005e00 tx=0x7fe594005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3763ef6a282a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5817fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5817fa640 1 -- 172.21.15.129:0/941706383 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940056f0 con 0x55617a6d8c30 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5817fa640 1 -- 172.21.15.129:0/941706383 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe59400f790 con 0x55617a6d8c30 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/941706383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5940041f0 con 0x55617a6d8c30 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5817fa640 1 -- 172.21.15.129:0/941706383 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe594003b80 con 0x55617a6d8c30 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5a2dc0740 10 client.16035 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/941706383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55617a4147d0 con 0x55617a6d8c30 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5817fa640 1 -- 172.21.15.129:0/941706383 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940041f0 con 0x55617a6d8c30 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5817fa640 1 -- 172.21.15.129:0/941706383 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fe59401e070 con 0x55617a6d8c30 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5a2dc0740 10 client.16035 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5a2dc0740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5a2dc0740 20 client.16035 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.313+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55617a6eeaf0 0x55617a6f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.314+0000 7fe5a2dc0740 4 client.16035 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:25:42.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.314+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/941706383 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55617a646990 con 0x55617a6eeaf0 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.314+0000 7fe59dd62640 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55617a6eeaf0 0x55617a6f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.314+0000 7fe5817fa640 10 client.16035 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.314+0000 7fe59dd62640 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55617a6eeaf0 0x55617a6f0f50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5657d1d3e62910 server_cookie=7a15135692fe9ff9 in_seq=0 out_seq=0 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5817fa640 1 -- 172.21.15.129:0/941706383 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55617a646990 con 0x55617a6eeaf0 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5817fa640 10 client.16035 handle_command_reply: tid=0 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 client.16035 shutdown 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/941706383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 msgr2=0x55617a6e2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6e2e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe594005e00 tx=0x7fe594005d50 comp rx=0 tx=0).stop 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/941706383 shutdown_connections 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55617a6eeaf0 0x55617a6f0f50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6da7c0 0x55617a6ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/941706383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6d8c30 0x55617a6e2e40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5817fa640 0 client.16035 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:42.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/941706383 >> 172.21.15.129:0/941706383 conn(0x55617a615910 msgr2=0x55617a6cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:42.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/941706383 shutdown_connections 2024-04-26T21:25:42.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/941706383 wait complete. 2024-04-26T21:25:42.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.316+0000 7fe5a2dc0740 20 client.16035 trim_cache size 0 max 16384 2024-04-26T21:25:42.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.329+0000 7fe5a2dc0740 1 Processor -- start 2024-04-26T21:25:42.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.329+0000 7fe5a2dc0740 1 -- start start 2024-04-26T21:25:42.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.329+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d8c30 0x55617a6e32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.329+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d9e00 0x55617a6ea380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.329+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.329+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981a5b70 con 0x55617a6da7c0 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981a83f0 con 0x55617a6d9e00 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981a61e0 con 0x55617a6d8c30 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43566/0 (socket says 172.21.15.129:43566) 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 -- 172.21.15.129:0/3858263154 learned_addr learned my addr 172.21.15.129:0/3858263154 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59dd62640 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d9e00 0x55617a6ea380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d8c30 0x55617a6e32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 -- 172.21.15.129:0/3858263154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d8c30 msgr2=0x55617a6e32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d8c30 0x55617a6e32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 -- 172.21.15.129:0/3858263154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d9e00 msgr2=0x55617a6ea380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d9e00 0x55617a6ea380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 -- 172.21.15.129:0/3858263154 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617a6f15d0 con 0x55617a6da7c0 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59dd62640 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d9e00 0x55617a6ea380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe59d561640 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6cfa00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe594003980 tx=0x7fe59400ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bf4cf4edaf5ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe5837fe640 1 -- 172.21.15.129:0/3858263154 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940132b0 con 0x55617a6da7c0 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe5837fe640 1 -- 172.21.15.129:0/3858263154 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe594003d20 con 0x55617a6da7c0 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.330+0000 7fe5837fe640 1 -- 172.21.15.129:0/3858263154 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59401fa10 con 0x55617a6da7c0 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3858263154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 msgr2=0x55617a6cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6cfa00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe594003980 tx=0x7fe59400ffd0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3858263154 shutdown_connections 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d9e00 0x55617a6ea380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6cfa00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/3858263154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d8c30 0x55617a6e32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3858263154 >> 172.21.15.129:0/3858263154 conn(0x55617a615910 msgr2=0x55617a6d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3858263154 shutdown_connections 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/3858263154 wait complete. 2024-04-26T21:25:42.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 -- start start 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.331+0000 7fe5a2dc0740 1 Processor -- start 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d8c30 0x55617a6e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981a68b0 con 0x55617a6da7c0 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981a7d20 con 0x55617a6d8c30 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981a6f80 con 0x55617a6d9e00 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 10 client.0 _ll_register_callbacks cb 0x7fe588009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6e8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43576/0 (socket says 172.21.15.129:43576) 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 -- 172.21.15.129:0/2748213342 learned_addr learned my addr 172.21.15.129:0/2748213342 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d8c30 0x55617a6e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59dd62640 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 -- 172.21.15.129:0/2748213342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 msgr2=0x55617a6e7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6e7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 -- 172.21.15.129:0/2748213342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d8c30 msgr2=0x55617a6e7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d8c30 0x55617a6e7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 -- 172.21.15.129:0/2748213342 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617a6cb8b0 con 0x55617a6da7c0 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe59d561640 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6e8f70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe594006010 tx=0x7fe594000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41d519e1140379d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5817fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5817fa640 1 -- 172.21.15.129:0/2748213342 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59403b030 con 0x55617a6da7c0 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5817fa640 1 -- 172.21.15.129:0/2748213342 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe5940040d0 con 0x55617a6da7c0 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/2748213342 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe594003d20 con 0x55617a6da7c0 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5817fa640 1 -- 172.21.15.129:0/2748213342 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe59400f9a0 con 0x55617a6da7c0 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 10 client.16041 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.332+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/2748213342 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55617a6ea670 con 0x55617a6da7c0 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe5817fa640 1 -- 172.21.15.129:0/2748213342 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe594003d20 con 0x55617a6da7c0 2024-04-26T21:25:42.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe5817fa640 1 -- 172.21.15.129:0/2748213342 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fe59401e040 con 0x55617a6da7c0 2024-04-26T21:25:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe5a2dc0740 10 client.16041 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe5a2dc0740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe5a2dc0740 20 client.16041 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55617a7557a0 0x55617a757b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe5a2dc0740 4 client.16041 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:25:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/2748213342 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55617a640800 con 0x55617a7557a0 2024-04-26T21:25:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe59dd62640 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55617a7557a0 0x55617a757b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe5817fa640 10 client.16041 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:42.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.333+0000 7fe59dd62640 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55617a7557a0 0x55617a757b90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12508536ff71fb56 server_cookie=35bb2fefd1a465ca in_seq=0 out_seq=0 2024-04-26T21:25:42.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5817fa640 1 -- 172.21.15.129:0/2748213342 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55617a640800 con 0x55617a7557a0 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5817fa640 10 client.16041 handle_command_reply: tid=0 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 client.16041 shutdown 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/2748213342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 msgr2=0x55617a6e8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6e8f70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe594006010 tx=0x7fe594000bc0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/2748213342 shutdown_connections 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55617a7557a0 0x55617a757b90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55617a6da7c0 0x55617a6e8f70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5817fa640 0 client.16041 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55617a6d9e00 0x55617a6e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 --2- 172.21.15.129:0/2748213342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55617a6d8c30 0x55617a6e7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/2748213342 >> 172.21.15.129:0/2748213342 conn(0x55617a615910 msgr2=0x55617a6e3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/2748213342 shutdown_connections 2024-04-26T21:25:42.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 1 -- 172.21.15.129:0/2748213342 wait complete. 2024-04-26T21:25:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.337+0000 7fe5a2dc0740 20 client.16041 trim_cache size 0 max 16384 2024-04-26T21:25:42.343 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:25:42.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 -- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe5840a3c40 msgr2=0x7fe5840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe5840a3c40 0x7fe5840a6100 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe5900bdc30 tx=0x7fe5900907e0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 -- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 msgr2=0x7fe5981a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 0x7fe5981a0150 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe590031a20 tx=0x7fe5900317a0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 -- 172.21.15.129:0/155298295 shutdown_connections 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fe5840a3c40 0x7fe5840a6100 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe598104520 0x7fe5981a0150 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe598103b60 0x7fe598199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 --2- 172.21.15.129:0/155298295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102970 0x7fe598198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 -- 172.21.15.129:0/155298295 >> 172.21.15.129:0/155298295 conn(0x7fe5980fdee0 msgr2=0x7fe598106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 -- 172.21.15.129:0/155298295 shutdown_connections 2024-04-26T21:25:42.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:42.338+0000 7fe59ffc6640 1 -- 172.21.15.129:0/155298295 wait complete. 2024-04-26T21:25:42.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:42 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1657350132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:25:42.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:42 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1657350132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:25:42.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:42 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1657350132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:25:43.350 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:25:43.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:25:43.350 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:25:43.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 -- 172.21.15.129:0/3626127067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0106760 msgr2=0x7fbdd010cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/3626127067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0106760 0x7fbdd010cff0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009ca0 tx=0x7fbdc802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 -- 172.21.15.129:0/3626127067 shutdown_connections 2024-04-26T21:25:43.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/3626127067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0106760 0x7fbdd010cff0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/3626127067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdd0105da0 0x7fbdd0106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/3626127067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdd0104bb0 0x7fbdd0104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 -- 172.21.15.129:0/3626127067 >> 172.21.15.129:0/3626127067 conn(0x7fbdd0100120 msgr2=0x7fbdd0102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 -- 172.21.15.129:0/3626127067 shutdown_connections 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 -- 172.21.15.129:0/3626127067 wait complete. 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.457+0000 7fbdd87ac640 1 Processor -- start 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd87ac640 1 -- start start 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd87ac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0104bb0 0x7fbdd019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd87ac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdd0105da0 0x7fbdd019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd87ac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdd0106760 0x7fbdd01a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd87ac640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01106e0 con 0x7fbdd0104bb0 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd87ac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0110560 con 0x7fbdd0105da0 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd87ac640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0110860 con 0x7fbdd0106760 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0104bb0 0x7fbdd019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0104bb0 0x7fbdd019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51964/0 (socket says 172.21.15.129:51964) 2024-04-26T21:25:43.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6548640 1 -- 172.21.15.129:0/2629835570 learned_addr learned my addr 172.21.15.129:0/2629835570 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd5d47640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdd0105da0 0x7fbdd019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6d49640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdd0106760 0x7fbdd01a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6548640 1 -- 172.21.15.129:0/2629835570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdd0106760 msgr2=0x7fbdd01a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdd0106760 0x7fbdd01a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6548640 1 -- 172.21.15.129:0/2629835570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdd0105da0 msgr2=0x7fbdd019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdd0105da0 0x7fbdd019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6548640 1 -- 172.21.15.129:0/2629835570 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd01a2980 con 0x7fbdd0104bb0 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0104bb0 0x7fbdd019ad60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000e9c0 tx=0x7fbdc000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdc77fe640 1 -- 172.21.15.129:0/2629835570 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000cd80 con 0x7fbdd0104bb0 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdc77fe640 1 -- 172.21.15.129:0/2629835570 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc00035f0 con 0x7fbdd0104bb0 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdc77fe640 1 -- 172.21.15.129:0/2629835570 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0010690 con 0x7fbdd0104bb0 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.458+0000 7fbdd87ac640 1 -- 172.21.15.129:0/2629835570 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd01a2c70 con 0x7fbdd0104bb0 2024-04-26T21:25:43.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.459+0000 7fbdd87ac640 1 -- 172.21.15.129:0/2629835570 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd01a31c0 con 0x7fbdd0104bb0 2024-04-26T21:25:43.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.462+0000 7fbddb59b740 1 Processor -- start 2024-04-26T21:25:43.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.462+0000 7fbddb59b740 1 -- start start 2024-04-26T21:25:43.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.462+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.462+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.462+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cdac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.462+0000 7fbddb59b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56456cd00b00 con 0x56456cda40c0 2024-04-26T21:25:43.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.462+0000 7fbddb59b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56456cc9c920 con 0x56456cda9c20 2024-04-26T21:25:43.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.462+0000 7fbddb59b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56456ccfcac0 con 0x56456cda4ab0 2024-04-26T21:25:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.464+0000 7fbdc77fe640 1 -- 172.21.15.129:0/2629835570 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbdc00037a0 con 0x7fbdd0104bb0 2024-04-26T21:25:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.465+0000 7fbdc77fe640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbdbc0a3cc0 0x7fbdbc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.465+0000 7fbdc77fe640 1 -- 172.21.15.129:0/2629835570 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fbdc00d0a10 con 0x7fbdd0104bb0 2024-04-26T21:25:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd5d47640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cdac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:38014/0 (socket says 172.21.15.129:38014) 2024-04-26T21:25:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6548640 1 -- 172.21.15.129:0/2714648446 learned_addr learned my addr 172.21.15.129:0/2714648446 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6d49640 1 --2- 172.21.15.129:0/2714648446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6d49640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbdbc0a3cc0 0x7fbdbc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6548640 1 -- 172.21.15.129:0/2714648446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 msgr2=0x56456cdac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2714648446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cdac0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6548640 1 -- 172.21.15.129:0/2714648446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 msgr2=0x56456cda44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2714648446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6548640 1 -- 172.21.15.129:0/2714648446 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456cada7e0 con 0x56456cda4ab0 2024-04-26T21:25:43.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.466+0000 7fbdd6d49640 1 --2- 172.21.15.129:0/2714648446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:25:43.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.467+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2714648446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda95b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00a5260 tx=0x7fbdc009c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69d829ae35fda50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.467+0000 7fbdd6d49640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbdbc0a3cc0 0x7fbdbc0a6180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x56456cda5310 tx=0x7fbdc80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.467+0000 7fbdc4ff9640 1 -- 172.21.15.129:0/2714648446 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00b6070 con 0x56456cda4ab0 2024-04-26T21:25:43.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.467+0000 7fbdc4ff9640 1 -- 172.21.15.129:0/2714648446 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc009de50 con 0x56456cda4ab0 2024-04-26T21:25:43.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.467+0000 7fbdc4ff9640 1 -- 172.21.15.129:0/2714648446 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0099190 con 0x56456cda4ab0 2024-04-26T21:25:43.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.468+0000 7fbddb59b740 1 -- 172.21.15.129:0/2714648446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 msgr2=0x56456cda95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.468+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2714648446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda95b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00a5260 tx=0x7fbdc009c480 comp rx=0 tx=0).stop 2024-04-26T21:25:43.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.469+0000 7fbddb59b740 1 -- 172.21.15.129:0/2714648446 shutdown_connections 2024-04-26T21:25:43.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.469+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2714648446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cdac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.469+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2714648446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda95b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.469+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2714648446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.469+0000 7fbddb59b740 1 -- 172.21.15.129:0/2714648446 >> 172.21.15.129:0/2714648446 conn(0x56456cd96f20 msgr2=0x56456cd97320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.470+0000 7fbddb59b740 1 -- 172.21.15.129:0/2714648446 shutdown_connections 2024-04-26T21:25:43.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.470+0000 7fbddb59b740 1 -- 172.21.15.129:0/2714648446 wait complete. 2024-04-26T21:25:43.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.470+0000 7fbddb59b740 1 -- start start 2024-04-26T21:25:43.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.470+0000 7fbddb59b740 1 Processor -- start 2024-04-26T21:25:43.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda40c0 0x56456cd9d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda4ab0 0x56456cd9d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda9c20 0x56456cda2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbddb59b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56456cd9b950 con 0x56456cda4ab0 2024-04-26T21:25:43.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbddb59b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56456cc9a5d0 con 0x56456cda40c0 2024-04-26T21:25:43.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbddb59b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56456cd9bad0 con 0x56456cda9c20 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbddb59b740 10 client.0 _ll_register_callbacks cb 0x56456cc88080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd5d47640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda4ab0 0x56456cd9d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda40c0 0x56456cd9d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd6d49640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda9c20 0x56456cda2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd5d47640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda4ab0 0x56456cd9d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51986/0 (socket says 172.21.15.129:51986) 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda40c0 0x56456cd9d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:49200/0 (socket says 172.21.15.129:49200) 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd5d47640 1 -- 172.21.15.129:0/2951539573 learned_addr learned my addr 172.21.15.129:0/2951539573 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd5d47640 1 -- 172.21.15.129:0/2951539573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda9c20 msgr2=0x56456cda2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd5d47640 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda9c20 0x56456cda2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd5d47640 1 -- 172.21.15.129:0/2951539573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda40c0 msgr2=0x56456cd9d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd5d47640 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda40c0 0x56456cd9d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd5d47640 1 -- 172.21.15.129:0/2951539573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456cb3af70 con 0x56456cda4ab0 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd6d49640 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda9c20 0x56456cda2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda40c0 0x56456cd9d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdd5d47640 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda4ab0 0x56456cd9d920 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc011660 tx=0x7fbdcc011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04783052cbac42a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.471+0000 7fbdb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2951539573 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc004220 con 0x56456cda4ab0 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2951539573 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fbdcc0043c0 con 0x56456cda4ab0 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbddb59b740 1 -- 172.21.15.129:0/2951539573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc009de50 con 0x56456cda4ab0 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2951539573 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbdcc011e60 con 0x56456cda4ab0 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbddb59b740 10 client.16053 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbddb59b740 1 -- 172.21.15.129:0/2951539573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56456c9d2170 con 0x56456cda4ab0 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2951539573 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc009de50 con 0x56456cda4ab0 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2951539573 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fbdcc003bc0 con 0x56456cda4ab0 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbddb59b740 10 client.16053 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbddb59b740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:43.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbddb59b740 20 client.16053 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56456cdc2900 0x56456cdc4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbddb59b740 4 client.16053 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:25:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbddb59b740 1 -- 172.21.15.129:0/2951539573 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56456cd0e710 con 0x56456cdc2900 2024-04-26T21:25:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.472+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56456cdc2900 0x56456cdc4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.473+0000 7fbdb9ffb640 10 client.16053 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.473+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56456cdc2900 0x56456cdc4d80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ffae47661ab747 server_cookie=ddc20d6466f5c082 in_seq=0 out_seq=0 2024-04-26T21:25:43.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.476+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2951539573 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56456cd0e710 con 0x56456cdc2900 2024-04-26T21:25:43.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.476+0000 7fbdb9ffb640 10 client.16053 handle_command_reply: tid=0 2024-04-26T21:25:43.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.476+0000 7fbddb59b740 1 client.16053 shutdown 2024-04-26T21:25:43.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 -- 172.21.15.129:0/2951539573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda4ab0 msgr2=0x56456cd9d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda4ab0 0x56456cd9d920 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc011660 tx=0x7fbdcc011b20 comp rx=0 tx=0).stop 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 -- 172.21.15.129:0/2951539573 shutdown_connections 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56456cdc2900 0x56456cdc4d80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda9c20 0x56456cda2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda4ab0 0x56456cd9d920 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbdb9ffb640 0 client.16053 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2951539573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda40c0 0x56456cd9d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 -- 172.21.15.129:0/2951539573 >> 172.21.15.129:0/2951539573 conn(0x56456cd96f20 msgr2=0x56456cd960e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 -- 172.21.15.129:0/2951539573 shutdown_connections 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 1 -- 172.21.15.129:0/2951539573 wait complete. 2024-04-26T21:25:43.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.477+0000 7fbddb59b740 20 client.16053 trim_cache size 0 max 16384 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.494+0000 7fbddb59b740 1 Processor -- start 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.494+0000 7fbddb59b740 1 -- start start 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.494+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.494+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.494+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cd9bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.494+0000 7fbddb59b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01a5be0 con 0x56456cda40c0 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.494+0000 7fbddb59b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01a8460 con 0x56456cda9c20 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.494+0000 7fbddb59b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01a6250 con 0x56456cda4ab0 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51988/0 (socket says 172.21.15.129:51988) 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6548640 1 -- 172.21.15.129:0/3209250806 learned_addr learned my addr 172.21.15.129:0/3209250806 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6d49640 1 --2- 172.21.15.129:0/3209250806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cd9bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd5d47640 1 --2- 172.21.15.129:0/3209250806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6548640 1 -- 172.21.15.129:0/3209250806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 msgr2=0x56456cda6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6548640 1 --2- 172.21.15.129:0/3209250806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6548640 1 -- 172.21.15.129:0/3209250806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 msgr2=0x56456cd9bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6548640 1 --2- 172.21.15.129:0/3209250806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cd9bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6548640 1 -- 172.21.15.129:0/3209250806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456cda92d0 con 0x56456cda40c0 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdd6548640 1 --2- 172.21.15.129:0/3209250806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda6050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0018000 tx=0x7fbdc009c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532f900a62765e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdbbfff640 1 -- 172.21.15.129:0/3209250806 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00b6040 con 0x56456cda40c0 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdbbfff640 1 -- 172.21.15.129:0/3209250806 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc009de50 con 0x56456cda40c0 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.495+0000 7fbdbbfff640 1 -- 172.21.15.129:0/3209250806 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00a5300 con 0x56456cda40c0 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 -- 172.21.15.129:0/3209250806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 msgr2=0x56456cda6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 --2- 172.21.15.129:0/3209250806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda6050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0018000 tx=0x7fbdc009c5b0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 -- 172.21.15.129:0/3209250806 shutdown_connections 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 --2- 172.21.15.129:0/3209250806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cd9bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 --2- 172.21.15.129:0/3209250806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cda6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 --2- 172.21.15.129:0/3209250806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cda6050 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 -- 172.21.15.129:0/3209250806 >> 172.21.15.129:0/3209250806 conn(0x56456cd96f20 msgr2=0x56456cd97300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 -- 172.21.15.129:0/3209250806 shutdown_connections 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 -- 172.21.15.129:0/3209250806 wait complete. 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 -- start start 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 Processor -- start 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cdd35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cdd3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cdd4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01a6920 con 0x56456cda40c0 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01a7d90 con 0x56456cda9c20 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01a6ff0 con 0x56456cda4ab0 2024-04-26T21:25:43.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbddb59b740 10 client.0 _ll_register_callbacks cb 0x7fbdcc00ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cdd35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbdd6548640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cdd35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51992/0 (socket says 172.21.15.129:51992) 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbdd6548640 1 -- 172.21.15.129:0/2159260684 learned_addr learned my addr 172.21.15.129:0/2159260684 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbdd5d47640 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cdd3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.496+0000 7fbdd6d49640 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cdd4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdd6548640 1 -- 172.21.15.129:0/2159260684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 msgr2=0x56456cdd3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cdd3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdd6548640 1 -- 172.21.15.129:0/2159260684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 msgr2=0x56456cdd4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cdd4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdd6548640 1 -- 172.21.15.129:0/2159260684 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456cdd5100 con 0x56456cda40c0 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdd6548640 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cdd35c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c6a0 tx=0x7fbdc009cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e550aae290d773e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:25:43.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2159260684 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc009ae20 con 0x56456cda40c0 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2159260684 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fbdc00a5300 con 0x56456cda40c0 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbddb59b740 1 -- 172.21.15.129:0/2159260684 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc009de50 con 0x56456cda40c0 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2159260684 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbdc00a5500 con 0x56456cda40c0 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbddb59b740 10 client.16059 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbddb59b740 1 -- 172.21.15.129:0/2159260684 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56456cdd6440 con 0x56456cda40c0 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2159260684 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc009de50 con 0x56456cda40c0 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2159260684 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fbdc00bbce0 con 0x56456cda40c0 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbddb59b740 10 client.16059 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbddb59b740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbddb59b740 20 client.16059 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56456ce28aa0 0x56456ce2ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbddb59b740 4 client.16059 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.497+0000 7fbddb59b740 1 -- 172.21.15.129:0/2159260684 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56456cd08580 con 0x56456ce28aa0 2024-04-26T21:25:43.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.498+0000 7fbdd5d47640 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56456ce28aa0 0x56456ce2ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.498+0000 7fbdd5d47640 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56456ce28aa0 0x56456ce2ae90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3279be4ee67cda server_cookie=a87a815c639e743f in_seq=0 out_seq=0 2024-04-26T21:25:43.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.498+0000 7fbdb9ffb640 10 client.16059 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:43.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.513+0000 7fbdb9ffb640 1 -- 172.21.15.129:0/2159260684 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56456cd08580 con 0x56456ce28aa0 2024-04-26T21:25:43.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.513+0000 7fbdb9ffb640 10 client.16059 handle_command_reply: tid=0 2024-04-26T21:25:43.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.513+0000 7fbddb59b740 1 client.16059 shutdown 2024-04-26T21:25:43.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.513+0000 7fbddb59b740 1 -- 172.21.15.129:0/2159260684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 msgr2=0x56456cdd35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.513+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cdd35c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c6a0 tx=0x7fbdc009cca0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbddb59b740 1 -- 172.21.15.129:0/2159260684 shutdown_connections 2024-04-26T21:25:43.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56456ce28aa0 0x56456ce2ae90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56456cda9c20 0x56456cdd4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdb9ffb640 0 client.16059 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56456cda4ab0 0x56456cdd3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbddb59b740 1 --2- 172.21.15.129:0/2159260684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56456cda40c0 0x56456cdd35c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbddb59b740 1 -- 172.21.15.129:0/2159260684 >> 172.21.15.129:0/2159260684 conn(0x56456cd96f20 msgr2=0x56456cd9bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbddb59b740 1 -- 172.21.15.129:0/2159260684 shutdown_connections 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbddb59b740 1 -- 172.21.15.129:0/2159260684 wait complete. 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbddb59b740 20 client.16059 trim_cache size 0 max 16384 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:25:43.516 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "03907666-3f69-4c87-9e80-92747387b9a6", 2024-04-26T21:25:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:25:43.517 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:25:43.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 -- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbdbc0a3cc0 msgr2=0x7fbdbc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbdbc0a3cc0 0x7fbdbc0a6180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x56456cda5310 tx=0x7fbdc80095c0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 -- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0104bb0 msgr2=0x7fbdd019ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0104bb0 0x7fbdd019ad60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000e9c0 tx=0x7fbdc000ee80 comp rx=0 tx=0).stop 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 -- 172.21.15.129:0/2629835570 shutdown_connections 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fbdbc0a3cc0 0x7fbdbc0a6180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdd0105da0 0x7fbdd019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdd0106760 0x7fbdd01a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 --2- 172.21.15.129:0/2629835570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdd0104bb0 0x7fbdd019ad60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 -- 172.21.15.129:0/2629835570 >> 172.21.15.129:0/2629835570 conn(0x7fbdd0100120 msgr2=0x7fbdd0109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 -- 172.21.15.129:0/2629835570 shutdown_connections 2024-04-26T21:25:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.514+0000 7fbdd87ac640 1 -- 172.21.15.129:0/2629835570 wait complete. 2024-04-26T21:25:43.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:25:43.526 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:25:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 -- 172.21.15.129:0/621197237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 msgr2=0x7f90a410cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 --2- 172.21.15.129:0/621197237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 0x7f90a410cfd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f909c009ca0 tx=0x7f909c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 -- 172.21.15.129:0/621197237 shutdown_connections 2024-04-26T21:25:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 --2- 172.21.15.129:0/621197237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 0x7f90a410cfd0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 --2- 172.21.15.129:0/621197237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a4105d80 0x7f90a4106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 --2- 172.21.15.129:0/621197237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a4104b90 0x7f90a4104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 -- 172.21.15.129:0/621197237 >> 172.21.15.129:0/621197237 conn(0x7f90a41000e0 msgr2=0x7f90a4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 -- 172.21.15.129:0/621197237 shutdown_connections 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 -- 172.21.15.129:0/621197237 wait complete. 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 Processor -- start 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 -- start start 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a4104b90 0x7f90a419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a4105d80 0x7f90a419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 0x7f90a41a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41106c0 con 0x7f90a4106740 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90a4110540 con 0x7f90a4105d80 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90ab083640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90a4110840 con 0x7f90a4104b90 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.629+0000 7f90a9620640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 0x7f90a41a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a4105d80 0x7f90a419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a9620640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 0x7f90a41a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52006/0 (socket says 172.21.15.129:52006) 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a9620640 1 -- 172.21.15.129:0/3981426516 learned_addr learned my addr 172.21.15.129:0/3981426516 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a4104b90 0x7f90a419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a3fff640 1 -- 172.21.15.129:0/3981426516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a4104b90 msgr2=0x7f90a419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a3fff640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a4104b90 0x7f90a419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a3fff640 1 -- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 msgr2=0x7f90a41a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a3fff640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 0x7f90a41a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a3fff640 1 -- 172.21.15.129:0/3981426516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a41a2980 con 0x7f90a4105d80 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a9620640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 0x7f90a41a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a3fff640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a4105d80 0x7f90a419b2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f909000d720 tx=0x7f909000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a1ffb640 1 -- 172.21.15.129:0/3981426516 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090017070 con 0x7f90a4105d80 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90ab083640 1 -- 172.21.15.129:0/3981426516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a41a2c70 con 0x7f90a4105d80 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a1ffb640 1 -- 172.21.15.129:0/3981426516 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f909000de30 con 0x7f90a4105d80 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90a1ffb640 1 -- 172.21.15.129:0/3981426516 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90900047a0 con 0x7f90a4105d80 2024-04-26T21:25:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.630+0000 7f90ab083640 1 -- 172.21.15.129:0/3981426516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a41a31c0 con 0x7f90a4105d80 2024-04-26T21:25:43.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.632+0000 7f90a1ffb640 1 -- 172.21.15.129:0/3981426516 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f90900099d0 con 0x7f90a4105d80 2024-04-26T21:25:43.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90adda1740 1 Processor -- start 2024-04-26T21:25:43.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90adda1740 1 -- start start 2024-04-26T21:25:43.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90adda1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfadee0 0x55998bfae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90adda1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfaf0b0 0x55998bfaf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90adda1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 0x55998bfb6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90adda1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55998bf12d80 con 0x55998bfadee0 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90adda1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55998becc8b0 con 0x55998bfaf0b0 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90adda1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55998bf0ed40 con 0x55998bfafa70 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a9620640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfadee0 0x55998bfae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a1ffb640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f90940a3c40 0x7f90940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a9620640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfadee0 0x55998bfae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52022/0 (socket says 172.21.15.129:52022) 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a9620640 1 -- 172.21.15.129:0/1286419209 learned_addr learned my addr 172.21.15.129:0/1286419209 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/1286419209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfaf0b0 0x55998bfaf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a3fff640 1 --2- 172.21.15.129:0/1286419209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 0x55998bfb6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a9620640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f90940a3c40 0x7f90940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a1ffb640 1 -- 172.21.15.129:0/3981426516 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f90900d01c0 con 0x7f90a4105d80 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a8e1f640 1 -- 172.21.15.129:0/1286419209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 msgr2=0x55998bfb6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/1286419209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 0x55998bfb6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a8e1f640 1 -- 172.21.15.129:0/1286419209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfadee0 msgr2=0x55998bfae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/1286419209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfadee0 0x55998bfae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.633+0000 7f90a8e1f640 1 -- 172.21.15.129:0/1286419209 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55998be9da40 con 0x55998bfaf0b0 2024-04-26T21:25:43.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/1286419209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfaf0b0 0x55998bfaf530 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9098009bf0 tx=0x7f9098011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0ffee2080d8293b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90a9620640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f90940a3c40 0x7f90940a6100 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55998bfaefb0 tx=0x7f909c02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f908effd640 1 -- 172.21.15.129:0/1286419209 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098004310 con 0x55998bfaf0b0 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f908effd640 1 -- 172.21.15.129:0/1286419209 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90980044b0 con 0x55998bfaf0b0 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f908effd640 1 -- 172.21.15.129:0/1286419209 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980035f0 con 0x55998bfaf0b0 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90adda1740 1 -- 172.21.15.129:0/1286419209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfaf0b0 msgr2=0x55998bfaf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90adda1740 1 --2- 172.21.15.129:0/1286419209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfaf0b0 0x55998bfaf530 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9098009bf0 tx=0x7f9098011440 comp rx=0 tx=0).stop 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90adda1740 1 -- 172.21.15.129:0/1286419209 shutdown_connections 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90adda1740 1 --2- 172.21.15.129:0/1286419209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfaf0b0 0x55998bfaf530 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90adda1740 1 --2- 172.21.15.129:0/1286419209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 0x55998bfb6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90adda1740 1 --2- 172.21.15.129:0/1286419209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfadee0 0x55998bfae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90adda1740 1 -- 172.21.15.129:0/1286419209 >> 172.21.15.129:0/1286419209 conn(0x55998bfa93d0 msgr2=0x55998bfa97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.634+0000 7f90adda1740 1 -- 172.21.15.129:0/1286419209 shutdown_connections 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 1 -- 172.21.15.129:0/1286419209 wait complete. 2024-04-26T21:25:43.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 1 -- start start 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 1 Processor -- start 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 0x55998bfb8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfb8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 0x55998bfbf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55998bfb6a80 con 0x55998bfaf0b0 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55998be99d10 con 0x55998bfafa70 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55998bfb6c00 con 0x55998bfadee0 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90adda1740 10 client.0 _ll_register_callbacks cb 0x55998bec6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfb8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfb8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52038/0 (socket says 172.21.15.129:52038) 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90a3fff640 1 -- 172.21.15.129:0/2922065660 learned_addr learned my addr 172.21.15.129:0/2922065660 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90a9620640 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 0x55998bfbf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.635+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 0x55998bfb8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90a3fff640 1 -- 172.21.15.129:0/2922065660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 msgr2=0x55998bfb8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90a3fff640 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 0x55998bfb8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90a3fff640 1 -- 172.21.15.129:0/2922065660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 msgr2=0x55998bfbf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90a3fff640 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 0x55998bfbf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90a3fff640 1 -- 172.21.15.129:0/2922065660 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55998bcee7d0 con 0x55998bfaf0b0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90a3fff640 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfb8870 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9090098ca0 tx=0x7f9090021f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c86210141ab68f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f9083fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f9083fff640 1 -- 172.21.15.129:0/2922065660 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909009c9d0 con 0x55998bfaf0b0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f9083fff640 1 -- 172.21.15.129:0/2922065660 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9090098940 con 0x55998bfaf0b0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90adda1740 1 -- 172.21.15.129:0/2922065660 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90980044b0 con 0x55998bfaf0b0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f9083fff640 1 -- 172.21.15.129:0/2922065660 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9090020200 con 0x55998bfaf0b0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90adda1740 10 client.16071 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90adda1740 1 -- 172.21.15.129:0/2922065660 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55998bc243f0 con 0x55998bfaf0b0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f9083fff640 1 -- 172.21.15.129:0/2922065660 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90980044b0 con 0x55998bfaf0b0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f9083fff640 1 -- 172.21.15.129:0/2922065660 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f90900cdd50 con 0x55998bfaf0b0 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90adda1740 10 client.16071 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:43.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90adda1740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:43.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90adda1740 20 client.16071 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:43.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90adda1740 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55998bfc3ec0 0x55998bfc6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90adda1740 4 client.16071 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:25:43.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.636+0000 7f90adda1740 1 -- 172.21.15.129:0/2922065660 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55998bf20990 con 0x55998bfc3ec0 2024-04-26T21:25:43.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.637+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55998bfc3ec0 0x55998bfc6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.637+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55998bfc3ec0 0x55998bfc6320 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74401b26686e4c1e server_cookie=8a0da478631d8d7c in_seq=0 out_seq=0 2024-04-26T21:25:43.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.637+0000 7f9083fff640 10 client.16071 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:43.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f9083fff640 1 -- 172.21.15.129:0/2922065660 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55998bf20990 con 0x55998bfc3ec0 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f9083fff640 10 client.16071 handle_command_reply: tid=0 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 client.16071 shutdown 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 -- 172.21.15.129:0/2922065660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 msgr2=0x55998bfb8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfb8870 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9090098ca0 tx=0x7f9090021f10 comp rx=0 tx=0).stop 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 -- 172.21.15.129:0/2922065660 shutdown_connections 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55998bfc3ec0 0x55998bfc6320 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 0x55998bfbf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfb8870 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f9083fff640 0 client.16071 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 --2- 172.21.15.129:0/2922065660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 0x55998bfb8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 -- 172.21.15.129:0/2922065660 >> 172.21.15.129:0/2922065660 conn(0x55998bfa93d0 msgr2=0x55998bfa8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 -- 172.21.15.129:0/2922065660 shutdown_connections 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 1 -- 172.21.15.129:0/2922065660 wait complete. 2024-04-26T21:25:43.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.648+0000 7f90adda1740 20 client.16071 trim_cache size 0 max 16384 2024-04-26T21:25:43.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.665+0000 7f90adda1740 1 Processor -- start 2024-04-26T21:25:43.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.665+0000 7f90adda1740 1 -- start start 2024-04-26T21:25:43.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.665+0000 7f90adda1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfadee0 0x55998bfa4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.665+0000 7f90adda1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.665+0000 7f90adda1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 0x55998bfacad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.665+0000 7f90adda1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41a9f20 con 0x55998bfaf0b0 2024-04-26T21:25:43.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.665+0000 7f90adda1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41ac7a0 con 0x55998bfadee0 2024-04-26T21:25:43.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.665+0000 7f90adda1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41aa590 con 0x55998bfafa70 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52040/0 (socket says 172.21.15.129:52040) 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a3fff640 1 -- 172.21.15.129:0/4221331119 learned_addr learned my addr 172.21.15.129:0/4221331119 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/4221331119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfadee0 0x55998bfa4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a3fff640 1 -- 172.21.15.129:0/4221331119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 msgr2=0x55998bfacad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a9620640 1 --2- 172.21.15.129:0/4221331119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 0x55998bfacad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a3fff640 1 --2- 172.21.15.129:0/4221331119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 0x55998bfacad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a3fff640 1 -- 172.21.15.129:0/4221331119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfadee0 msgr2=0x55998bfa4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a3fff640 1 --2- 172.21.15.129:0/4221331119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfadee0 0x55998bfa4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a3fff640 1 -- 172.21.15.129:0/4221331119 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55998bbe40f0 con 0x55998bfaf0b0 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f90a3fff640 1 --2- 172.21.15.129:0/4221331119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbf870 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f90900203c0 tx=0x7f9090005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c77e35ed4083f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f908e7fc640 1 -- 172.21.15.129:0/4221331119 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90900ba070 con 0x55998bfaf0b0 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f908e7fc640 1 -- 172.21.15.129:0/4221331119 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90900b5070 con 0x55998bfaf0b0 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.666+0000 7f908e7fc640 1 -- 172.21.15.129:0/4221331119 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909009c2d0 con 0x55998bfaf0b0 2024-04-26T21:25:43.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 -- 172.21.15.129:0/4221331119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 msgr2=0x55998bfbf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 --2- 172.21.15.129:0/4221331119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbf870 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f90900203c0 tx=0x7f9090005c10 comp rx=0 tx=0).stop 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 -- 172.21.15.129:0/4221331119 shutdown_connections 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 --2- 172.21.15.129:0/4221331119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfafa70 0x55998bfacad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 --2- 172.21.15.129:0/4221331119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbf870 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 --2- 172.21.15.129:0/4221331119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfadee0 0x55998bfa4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 -- 172.21.15.129:0/4221331119 >> 172.21.15.129:0/4221331119 conn(0x55998bfa93d0 msgr2=0x55998bfac3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 -- 172.21.15.129:0/4221331119 shutdown_connections 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 -- 172.21.15.129:0/4221331119 wait complete. 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 -- start start 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.667+0000 7f90adda1740 1 Processor -- start 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90adda1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 0x55998bfbd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90adda1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90adda1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 0x55998bfbea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90adda1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41aac60 con 0x55998bfaf0b0 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90adda1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41ac0d0 con 0x55998bfafa70 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90adda1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41ab330 con 0x55998bfadee0 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90adda1740 10 client.0 _ll_register_callbacks cb 0x7f9090098640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52044/0 (socket says 172.21.15.129:52044) 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a3fff640 1 -- 172.21.15.129:0/2141254191 learned_addr learned my addr 172.21.15.129:0/2141254191 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a9620640 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 0x55998bfbea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 0x55998bfbd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a3fff640 1 -- 172.21.15.129:0/2141254191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 msgr2=0x55998bfbd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a3fff640 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 0x55998bfbd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a3fff640 1 -- 172.21.15.129:0/2141254191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 msgr2=0x55998bfbea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a3fff640 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 0x55998bfbea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a3fff640 1 -- 172.21.15.129:0/2141254191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55998bed34b0 con 0x55998bfaf0b0 2024-04-26T21:25:43.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90a3fff640 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbda70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f909009b740 tx=0x7f90900cdab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5169b3101f304cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f9083fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f9083fff640 1 -- 172.21.15.129:0/2141254191 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90900ba050 con 0x55998bfaf0b0 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f9083fff640 1 -- 172.21.15.129:0/2141254191 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f90900cdd50 con 0x55998bfaf0b0 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f90adda1740 1 -- 172.21.15.129:0/2141254191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90900b5070 con 0x55998bfaf0b0 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.668+0000 7f9083fff640 1 -- 172.21.15.129:0/2141254191 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f909009c9d0 con 0x55998bfaf0b0 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90adda1740 10 client.16077 fetch_fsmap learned FSMap version 19 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90adda1740 1 -- 172.21.15.129:0/2141254191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55998c02a7b0 con 0x55998bfaf0b0 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f9083fff640 1 -- 172.21.15.129:0/2141254191 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90900b5070 con 0x55998bfaf0b0 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f9083fff640 1 -- 172.21.15.129:0/2141254191 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f9090098020 con 0x55998bfaf0b0 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90adda1740 10 client.16077 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90adda1740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90adda1740 20 client.16077 populate_metadata read hostname 'smithi129' 2024-04-26T21:25:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90adda1740 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55998c02afc0 0x55998c02d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:43.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90adda1740 4 client.16077 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:25:43.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90adda1740 1 -- 172.21.15.129:0/2141254191 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55998bf1a800 con 0x55998c02afc0 2024-04-26T21:25:43.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55998c02afc0 0x55998c02d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:43.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f90a8e1f640 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55998c02afc0 0x55998c02d3e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14b8853669ae5b0 server_cookie=ad08126225c03501 in_seq=0 out_seq=0 2024-04-26T21:25:43.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.669+0000 7f9083fff640 10 client.16077 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f9083fff640 1 -- 172.21.15.129:0/2141254191 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55998bf1a800 con 0x55998c02afc0 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f9083fff640 10 client.16077 handle_command_reply: tid=0 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 client.16077 shutdown 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 -- 172.21.15.129:0/2141254191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 msgr2=0x55998bfbda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbda70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f909009b740 tx=0x7f90900cdab0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 -- 172.21.15.129:0/2141254191 shutdown_connections 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55998c02afc0 0x55998c02d3e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55998bfafa70 0x55998bfbea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55998bfaf0b0 0x55998bfbda70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 --2- 172.21.15.129:0/2141254191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55998bfadee0 0x55998bfbd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 -- 172.21.15.129:0/2141254191 >> 172.21.15.129:0/2141254191 conn(0x55998bfa93d0 msgr2=0x55998bfacc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f9083fff640 0 client.16077 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:25:43.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 -- 172.21.15.129:0/2141254191 shutdown_connections 2024-04-26T21:25:43.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.688+0000 7f90adda1740 1 -- 172.21.15.129:0/2141254191 wait complete. 2024-04-26T21:25:43.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90adda1740 20 client.16077 trim_cache size 0 max 16384 2024-04-26T21:25:43.691 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:25:43.691 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (4484 inodes in the stack)", 2024-04-26T21:25:43.691 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:25:43.691 INFO:teuthology.orchestra.run.smithi129.stdout: "03907666-3f69-4c87-9e80-92747387b9a6": { 2024-04-26T21:25:43.691 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:25:43.692 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "03907666-3f69-4c87-9e80-92747387b9a6", 2024-04-26T21:25:43.692 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:25:43.692 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:25:43.692 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:25:43.692 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 -- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f90940a3c40 msgr2=0x7f90940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f90940a3c40 0x7f90940a6100 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55998bfaefb0 tx=0x7f909c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 -- 172.21.15.129:0/3981426516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a4105d80 msgr2=0x7f90a419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a4105d80 0x7f90a419b2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f909000d720 tx=0x7f909000dbe0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 -- 172.21.15.129:0/3981426516 shutdown_connections 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f90940a3c40 0x7f90940a6100 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a4105d80 0x7f90a419b2a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4106740 0x7f90a41a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 --2- 172.21.15.129:0/3981426516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a4104b90 0x7f90a419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 -- 172.21.15.129:0/3981426516 >> 172.21.15.129:0/3981426516 conn(0x7f90a41000e0 msgr2=0x7f90a4109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 -- 172.21.15.129:0/3981426516 shutdown_connections 2024-04-26T21:25:43.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:25:43.689+0000 7f90ab083640 1 -- 172.21.15.129:0/3981426516 wait complete. 2024-04-26T21:25:43.700 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:43 smithi129 ceph-mon[29237]: pgmap v582: 97 pgs: 97 active+clean; 505 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 560 op/s 2024-04-26T21:25:43.701 INFO:tasks.cephfs.filesystem:scrub status for tag:03907666-3f69-4c87-9e80-92747387b9a6 - {'path': '/', 'tag': '03907666-3f69-4c87-9e80-92747387b9a6', 'options': 'recursive,force'} 2024-04-26T21:25:43.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:43 smithi150 ceph-mon[32290]: pgmap v582: 97 pgs: 97 active+clean; 505 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 560 op/s 2024-04-26T21:25:43.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:43 smithi171 ceph-mon[33478]: pgmap v582: 97 pgs: 97 active+clean; 505 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 560 op/s 2024-04-26T21:25:44.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:44 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:25:44.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:44 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:25:44.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:44 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:25:44.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:44 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:25:44.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:44 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:25:44.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:44 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:25:44.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:44 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:25:44.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:44 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:25:44.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:44 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:25:45.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:45 smithi150 ceph-mon[32290]: pgmap v583: 97 pgs: 97 active+clean; 496 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 521 op/s 2024-04-26T21:25:45.892 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:45 smithi171 ceph-mon[33478]: pgmap v583: 97 pgs: 97 active+clean; 496 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 521 op/s 2024-04-26T21:25:45.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:45 smithi129 ceph-mon[29237]: pgmap v583: 97 pgs: 97 active+clean; 496 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 521 op/s 2024-04-26T21:25:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:47 smithi150 ceph-mon[32290]: pgmap v584: 97 pgs: 97 active+clean; 486 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 560 op/s 2024-04-26T21:25:47.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:47 smithi129 ceph-mon[29237]: pgmap v584: 97 pgs: 97 active+clean; 486 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 560 op/s 2024-04-26T21:25:47.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:47 smithi171 ceph-mon[33478]: pgmap v584: 97 pgs: 97 active+clean; 486 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 560 op/s 2024-04-26T21:25:49.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:49 smithi150 ceph-mon[32290]: pgmap v585: 97 pgs: 97 active+clean; 474 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 695 op/s 2024-04-26T21:25:49.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:49 smithi129 ceph-mon[29237]: pgmap v585: 97 pgs: 97 active+clean; 474 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 695 op/s 2024-04-26T21:25:49.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:49 smithi171 ceph-mon[33478]: pgmap v585: 97 pgs: 97 active+clean; 474 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 695 op/s 2024-04-26T21:25:51.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:51 smithi150 ceph-mon[32290]: pgmap v586: 97 pgs: 97 active+clean; 472 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 657 op/s 2024-04-26T21:25:51.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:51 smithi129 ceph-mon[29237]: pgmap v586: 97 pgs: 97 active+clean; 472 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 657 op/s 2024-04-26T21:25:51.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:51 smithi171 ceph-mon[33478]: pgmap v586: 97 pgs: 97 active+clean; 472 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 657 op/s 2024-04-26T21:25:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:53 smithi150 ceph-mon[32290]: pgmap v587: 97 pgs: 97 active+clean; 462 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 690 op/s 2024-04-26T21:25:53.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:53 smithi129 ceph-mon[29237]: pgmap v587: 97 pgs: 97 active+clean; 462 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 690 op/s 2024-04-26T21:25:53.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:53 smithi171 ceph-mon[33478]: pgmap v587: 97 pgs: 97 active+clean; 462 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 690 op/s 2024-04-26T21:25:55.886 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:55 smithi150 ceph-mon[32290]: pgmap v588: 97 pgs: 97 active+clean; 454 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 642 op/s 2024-04-26T21:25:55.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:55 smithi129 ceph-mon[29237]: pgmap v588: 97 pgs: 97 active+clean; 454 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 642 op/s 2024-04-26T21:25:55.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:55 smithi171 ceph-mon[33478]: pgmap v588: 97 pgs: 97 active+clean; 454 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 642 op/s 2024-04-26T21:25:57.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:57 smithi150 ceph-mon[32290]: pgmap v589: 97 pgs: 97 active+clean; 456 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 726 op/s 2024-04-26T21:25:57.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:57 smithi129 ceph-mon[29237]: pgmap v589: 97 pgs: 97 active+clean; 456 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 726 op/s 2024-04-26T21:25:57.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:57 smithi171 ceph-mon[33478]: pgmap v589: 97 pgs: 97 active+clean; 456 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 726 op/s 2024-04-26T21:25:59.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:25:59 smithi150 ceph-mon[32290]: pgmap v590: 97 pgs: 97 active+clean; 451 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 842 op/s 2024-04-26T21:25:59.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:25:59 smithi129 ceph-mon[29237]: pgmap v590: 97 pgs: 97 active+clean; 451 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 842 op/s 2024-04-26T21:25:59.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:25:59 smithi171 ceph-mon[33478]: pgmap v590: 97 pgs: 97 active+clean; 451 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 842 op/s 2024-04-26T21:26:01.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:01 smithi150 ceph-mon[32290]: pgmap v591: 97 pgs: 97 active+clean; 429 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 794 op/s 2024-04-26T21:26:01.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:01 smithi129 ceph-mon[29237]: pgmap v591: 97 pgs: 97 active+clean; 429 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 794 op/s 2024-04-26T21:26:01.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:01 smithi171 ceph-mon[33478]: pgmap v591: 97 pgs: 97 active+clean; 429 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 794 op/s 2024-04-26T21:26:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:03 smithi150 ceph-mon[32290]: pgmap v592: 97 pgs: 97 active+clean; 407 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 944 op/s 2024-04-26T21:26:03.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:03 smithi129 ceph-mon[29237]: pgmap v592: 97 pgs: 97 active+clean; 407 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 944 op/s 2024-04-26T21:26:03.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:03 smithi171 ceph-mon[33478]: pgmap v592: 97 pgs: 97 active+clean; 407 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 944 op/s 2024-04-26T21:26:05.469 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:05 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:26:05.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:05 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:26:05.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:05 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:26:06.470 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:06 smithi129 ceph-mon[29237]: pgmap v593: 97 pgs: 97 active+clean; 400 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 985 op/s 2024-04-26T21:26:06.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:06 smithi171 ceph-mon[33478]: pgmap v593: 97 pgs: 97 active+clean; 400 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 985 op/s 2024-04-26T21:26:06.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:06 smithi150 ceph-mon[32290]: pgmap v593: 97 pgs: 97 active+clean; 400 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 985 op/s 2024-04-26T21:26:07.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:07 smithi150 ceph-mon[32290]: pgmap v594: 97 pgs: 97 active+clean; 396 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 1.09k op/s 2024-04-26T21:26:07.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:07 smithi129 ceph-mon[29237]: pgmap v594: 97 pgs: 97 active+clean; 396 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 1.09k op/s 2024-04-26T21:26:07.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:07 smithi171 ceph-mon[33478]: pgmap v594: 97 pgs: 97 active+clean; 396 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 1.09k op/s 2024-04-26T21:26:09.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:09 smithi150 ceph-mon[32290]: pgmap v595: 97 pgs: 97 active+clean; 384 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.08k op/s 2024-04-26T21:26:09.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:09 smithi129 ceph-mon[29237]: pgmap v595: 97 pgs: 97 active+clean; 384 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.08k op/s 2024-04-26T21:26:09.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:09 smithi171 ceph-mon[33478]: pgmap v595: 97 pgs: 97 active+clean; 384 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.08k op/s 2024-04-26T21:26:10.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:10 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:26:10.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:10 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:26:10.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:10 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:26:12.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:11 smithi150 ceph-mon[32290]: pgmap v596: 97 pgs: 97 active+clean; 368 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 930 op/s 2024-04-26T21:26:12.220 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:11 smithi129 ceph-mon[29237]: pgmap v596: 97 pgs: 97 active+clean; 368 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 930 op/s 2024-04-26T21:26:12.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:11 smithi171 ceph-mon[33478]: pgmap v596: 97 pgs: 97 active+clean; 368 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 930 op/s 2024-04-26T21:26:13.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:26:13.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:26:13.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 -- 172.21.15.129:0/3485680257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8105d40 msgr2=0x7fa6c8108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 --2- 172.21.15.129:0/3485680257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8105d40 0x7fa6c8108130 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009900 tx=0x7fa6bc02a410 comp rx=0 tx=0).stop 2024-04-26T21:26:13.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 -- 172.21.15.129:0/3485680257 shutdown_connections 2024-04-26T21:26:13.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 --2- 172.21.15.129:0/3485680257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c8108670 0x7fa6c810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 --2- 172.21.15.129:0/3485680257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8105d40 0x7fa6c8108130 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 --2- 172.21.15.129:0/3485680257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c8103410 0x7fa6c8105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 -- 172.21.15.129:0/3485680257 >> 172.21.15.129:0/3485680257 conn(0x7fa6c80fce20 msgr2=0x7fa6c80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 -- 172.21.15.129:0/3485680257 shutdown_connections 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 -- 172.21.15.129:0/3485680257 wait complete. 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 Processor -- start 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.804+0000 7fa6cce59640 1 -- start start 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6cce59640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c8103410 0x7fa6c8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6cce59640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c8105d40 0x7fa6c8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6cce59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8108670 0x7fa6c81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6cce59640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c810e4b0 con 0x7fa6c8108670 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6cce59640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c810e330 con 0x7fa6c8103410 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6cce59640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c810e630 con 0x7fa6c8105d40 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8108670 0x7fa6c81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c8103410 0x7fa6c8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8108670 0x7fa6c81a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51920/0 (socket says 172.21.15.129:51920) 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c8103410 0x7fa6c8198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60932/0 (socket says 172.21.15.129:60932) 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c6d9d640 1 -- 172.21.15.129:0/2498791393 learned_addr learned my addr 172.21.15.129:0/2498791393 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c8105d40 0x7fa6c8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c6d9d640 1 -- 172.21.15.129:0/2498791393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c8105d40 msgr2=0x7fa6c8199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c6d9d640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c8105d40 0x7fa6c8199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c6d9d640 1 -- 172.21.15.129:0/2498791393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c8103410 msgr2=0x7fa6c8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c6d9d640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c8103410 0x7fa6c8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c6d9d640 1 -- 172.21.15.129:0/2498791393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c81a0800 con 0x7fa6c8108670 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c659c640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c8103410 0x7fa6c8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6c6d9d640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8108670 0x7fa6c81a0190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000eb10 tx=0x7fa6c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6b77fe640 1 -- 172.21.15.129:0/2498791393 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0004b20 con 0x7fa6c8108670 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6cce59640 1 -- 172.21.15.129:0/2498791393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c81a0af0 con 0x7fa6c8108670 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6b77fe640 1 -- 172.21.15.129:0/2498791393 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c0004cc0 con 0x7fa6c8108670 2024-04-26T21:26:13.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6cce59640 1 -- 172.21.15.129:0/2498791393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c81a1010 con 0x7fa6c8108670 2024-04-26T21:26:13.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.805+0000 7fa6b77fe640 1 -- 172.21.15.129:0/2498791393 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0018630 con 0x7fa6c8108670 2024-04-26T21:26:13.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6b77fe640 1 -- 172.21.15.129:0/2498791393 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa6c0005000 con 0x7fa6c8108670 2024-04-26T21:26:13.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6cfb73740 1 Processor -- start 2024-04-26T21:26:13.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6cfb73740 1 -- start start 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528bee0 0x555ef528c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef528d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef5294300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555ef51f0d80 con 0x555ef528da70 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555ef51aa8b0 con 0x555ef528d0b0 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555ef51ecd40 con 0x555ef528bee0 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6b77fe640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa6b00a3c40 0x7fa6b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6c659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef528d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528bee0 0x555ef528c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6c659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef528d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:60944/0 (socket says 172.21.15.129:60944) 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6c659c640 1 -- 172.21.15.129:0/2225483675 learned_addr learned my addr 172.21.15.129:0/2225483675 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/2225483675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef5294300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.808+0000 7fa6c6d9d640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa6b00a3c40 0x7fa6b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6b77fe640 1 -- 172.21.15.129:0/2498791393 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa6c00d0ca0 con 0x7fa6c8108670 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6c659c640 1 -- 172.21.15.129:0/2225483675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528bee0 msgr2=0x555ef528c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6c659c640 1 --2- 172.21.15.129:0/2225483675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528bee0 0x555ef528c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6c659c640 1 -- 172.21.15.129:0/2225483675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 msgr2=0x555ef5294300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6c659c640 1 --2- 172.21.15.129:0/2225483675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef5294300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6c659c640 1 -- 172.21.15.129:0/2225483675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef517ba40 con 0x555ef528d0b0 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6c659c640 1 --2- 172.21.15.129:0/2225483675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef528d530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8010580 tx=0x7fa6b8010a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe52a5f7cf09441 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6c6d9d640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa6b00a3c40 0x7fa6b00a6100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa6c009cda0 tx=0x7fa6c000cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6b4ff9640 1 -- 172.21.15.129:0/2225483675 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8011760 con 0x555ef528d0b0 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6b4ff9640 1 -- 172.21.15.129:0/2225483675 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8011db0 con 0x555ef528d0b0 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.809+0000 7fa6b4ff9640 1 -- 172.21.15.129:0/2225483675 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b800ee00 con 0x555ef528d0b0 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2225483675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 msgr2=0x555ef528d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2225483675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef528d530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8010580 tx=0x7fa6b8010a40 comp rx=0 tx=0).stop 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2225483675 shutdown_connections 2024-04-26T21:26:13.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2225483675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef528d530 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2225483675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef5294300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2225483675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528bee0 0x555ef528c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2225483675 >> 172.21.15.129:0/2225483675 conn(0x555ef52873d0 msgr2=0x555ef52877d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2225483675 shutdown_connections 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2225483675 wait complete. 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 -- start start 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 Processor -- start 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5296390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef52968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528da70 0x555ef529d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555ef5294a80 con 0x555ef528bee0 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555ef5177d10 con 0x555ef528da70 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555ef5294c00 con 0x555ef528d0b0 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.810+0000 7fa6cfb73740 10 client.0 _ll_register_callbacks cb 0x555ef51a4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5296390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5296390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51944/0 (socket says 172.21.15.129:51944) 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/2193323080 learned_addr learned my addr 172.21.15.129:0/2193323080 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c659c640 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef52968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528da70 0x555ef529d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/2193323080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 msgr2=0x555ef52968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef52968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/2193323080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528da70 msgr2=0x555ef529d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528da70 0x555ef529d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/2193323080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef4fcc7d0 con 0x555ef528bee0 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c659c640 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef52968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5296390 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005c40 tx=0x7fa6bc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765788d2805305fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6adffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6adffb640 1 -- 172.21.15.129:0/2193323080 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc035040 con 0x555ef528bee0 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6adffb640 1 -- 172.21.15.129:0/2193323080 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa6bc0035f0 con 0x555ef528bee0 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2193323080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b8011db0 con 0x555ef528bee0 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6adffb640 1 -- 172.21.15.129:0/2193323080 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa6bc0037f0 con 0x555ef528bee0 2024-04-26T21:26:13.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6cfb73740 10 client.16089 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.811+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2193323080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555ef4f023f0 con 0x555ef528bee0 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6adffb640 1 -- 172.21.15.129:0/2193323080 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8011db0 con 0x555ef528bee0 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6adffb640 1 -- 172.21.15.129:0/2193323080 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fa6bc03a040 con 0x555ef528bee0 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6cfb73740 10 client.16089 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6cfb73740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6cfb73740 20 client.16089 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555ef52a1e40 0x555ef52a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6cfb73740 4 client.16089 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2193323080 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ef51fe990 con 0x555ef52a1e40 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6c659c640 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555ef52a1e40 0x555ef52a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6adffb640 10 client.16089 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.812+0000 7fa6c659c640 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555ef52a1e40 0x555ef52a42b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e83d23686adac53 server_cookie=41dc7973dbe2800a in_seq=0 out_seq=0 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6adffb640 1 -- 172.21.15.129:0/2193323080 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ef51fe990 con 0x555ef52a1e40 2024-04-26T21:26:13.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6adffb640 10 client.16089 handle_command_reply: tid=0 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 client.16089 shutdown 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2193323080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 msgr2=0x555ef5296390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5296390 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005c40 tx=0x7fa6bc0056d0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2193323080 shutdown_connections 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555ef52a1e40 0x555ef52a42b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528da70 0x555ef529d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef52968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6adffb640 0 client.16089 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/2193323080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5296390 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2193323080 >> 172.21.15.129:0/2193323080 conn(0x555ef52873d0 msgr2=0x555ef5286550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2193323080 shutdown_connections 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 1 -- 172.21.15.129:0/2193323080 wait complete. 2024-04-26T21:26:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.813+0000 7fa6cfb73740 20 client.16089 trim_cache size 0 max 16384 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6cfb73740 1 Processor -- start 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6cfb73740 1 -- start start 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5295fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef52964e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528da70 0x555ef528a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c81a7ed0 con 0x555ef528bee0 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c81aa750 con 0x555ef528d0b0 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c81a8540 con 0x555ef528da70 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5295fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5295fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51948/0 (socket says 172.21.15.129:51948) 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/1905562749 learned_addr learned my addr 172.21.15.129:0/1905562749 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c659c640 1 --2- 172.21.15.129:0/1905562749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef52964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/1905562749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528da70 msgr2=0x555ef528a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/1905562749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528da70 0x555ef528a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/1905562749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 msgr2=0x555ef52964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/1905562749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef52964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/1905562749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef4ec20f0 con 0x555ef528bee0 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/1905562749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5295fa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc032da0 tx=0x7fa6bc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10888168a1ae3451 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6affff640 1 -- 172.21.15.129:0/1905562749 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc035070 con 0x555ef528bee0 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6affff640 1 -- 172.21.15.129:0/1905562749 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6bc0035f0 con 0x555ef528bee0 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.831+0000 7fa6affff640 1 -- 172.21.15.129:0/1905562749 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc031990 con 0x555ef528bee0 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 -- 172.21.15.129:0/1905562749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 msgr2=0x555ef5295fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/1905562749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5295fa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc032da0 tx=0x7fa6bc005b50 comp rx=0 tx=0).stop 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 -- 172.21.15.129:0/1905562749 shutdown_connections 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/1905562749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528d0b0 0x555ef52964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/1905562749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528da70 0x555ef528a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/1905562749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528bee0 0x555ef5295fa0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 -- 172.21.15.129:0/1905562749 >> 172.21.15.129:0/1905562749 conn(0x555ef52873d0 msgr2=0x555ef5282480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 -- 172.21.15.129:0/1905562749 shutdown_connections 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 -- 172.21.15.129:0/1905562749 wait complete. 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 -- start start 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 Processor -- start 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528bee0 0x555ef53030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef5303600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef530a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c81a8c10 con 0x555ef528da70 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c81aa080 con 0x555ef528bee0 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.832+0000 7fa6cfb73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c81a92e0 con 0x555ef528d0b0 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6cfb73740 10 client.0 _ll_register_callbacks cb 0x7fa6bc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef530a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef530a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51966/0 (socket says 172.21.15.129:51966) 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/3504180018 learned_addr learned my addr 172.21.15.129:0/3504180018 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:13.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c659c640 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef5303600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528bee0 0x555ef53030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/3504180018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 msgr2=0x555ef5303600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef5303600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/3504180018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528bee0 msgr2=0x555ef53030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528bee0 0x555ef53030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 -- 172.21.15.129:0/3504180018 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef51b14b0 con 0x555ef528da70 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c659c640 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef5303600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6c5d9b640 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef530a5d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc02a930 tx=0x7fa6bc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c405b0919329911 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6adffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6adffb640 1 -- 172.21.15.129:0/3504180018 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc035040 con 0x555ef528da70 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6adffb640 1 -- 172.21.15.129:0/3504180018 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa6bc031990 con 0x555ef528da70 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6cfb73740 1 -- 172.21.15.129:0/3504180018 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6bc0035f0 con 0x555ef528da70 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6adffb640 1 -- 172.21.15.129:0/3504180018 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa6bc056470 con 0x555ef528da70 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6cfb73740 10 client.16095 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.833+0000 7fa6cfb73740 1 -- 172.21.15.129:0/3504180018 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555ef530bd00 con 0x555ef528da70 2024-04-26T21:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6adffb640 1 -- 172.21.15.129:0/3504180018 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6bc0035f0 con 0x555ef528da70 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6adffb640 1 -- 172.21.15.129:0/3504180018 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fa6bc061040 con 0x555ef528da70 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6cfb73740 10 client.16095 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6cfb73740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6cfb73740 20 client.16095 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555ef530c5c0 0x555ef530e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6cfb73740 4 client.16095 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6cfb73740 1 -- 172.21.15.129:0/3504180018 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ef51f8800 con 0x555ef530c5c0 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6c659c640 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555ef530c5c0 0x555ef530e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6c659c640 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555ef530c5c0 0x555ef530e9e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df90e805c6e0b0 server_cookie=8a4043d2d94aed59 in_seq=0 out_seq=0 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.834+0000 7fa6adffb640 10 client.16095 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6adffb640 1 -- 172.21.15.129:0/3504180018 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ef51f8800 con 0x555ef530c5c0 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6adffb640 10 client.16095 handle_command_reply: tid=0 2024-04-26T21:26:13.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 client.16095 shutdown 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 -- 172.21.15.129:0/3504180018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 msgr2=0x555ef530a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef530a5d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc02a930 tx=0x7fa6bc0056d0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 -- 172.21.15.129:0/3504180018 shutdown_connections 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x555ef530c5c0 0x555ef530e9e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ef528da70 0x555ef530a5d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555ef528d0b0 0x555ef5303600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6adffb640 0 client.16095 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 --2- 172.21.15.129:0/3504180018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ef528bee0 0x555ef53030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 -- 172.21.15.129:0/3504180018 >> 172.21.15.129:0/3504180018 conn(0x555ef52873d0 msgr2=0x555ef5295fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 -- 172.21.15.129:0/3504180018 shutdown_connections 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 1 -- 172.21.15.129:0/3504180018 wait complete. 2024-04-26T21:26:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cfb73740 20 client.16095 trim_cache size 0 max 16384 2024-04-26T21:26:13.844 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:26:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:26:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:26:13.844 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cce59640 1 -- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa6b00a3c40 msgr2=0x7fa6b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cce59640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa6b00a3c40 0x7fa6b00a6100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa6c009cda0 tx=0x7fa6c000cf70 comp rx=0 tx=0).stop 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cce59640 1 -- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8108670 msgr2=0x7fa6c81a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.835+0000 7fa6cce59640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8108670 0x7fa6c81a0190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000eb10 tx=0x7fa6c000c2d0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.836+0000 7fa6cce59640 1 -- 172.21.15.129:0/2498791393 shutdown_connections 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.836+0000 7fa6cce59640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa6b00a3c40 0x7fa6b00a6100 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.836+0000 7fa6cce59640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6c8108670 0x7fa6c81a0190 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.836+0000 7fa6cce59640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c8105d40 0x7fa6c8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.836+0000 7fa6cce59640 1 --2- 172.21.15.129:0/2498791393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c8103410 0x7fa6c8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.836+0000 7fa6cce59640 1 -- 172.21.15.129:0/2498791393 >> 172.21.15.129:0/2498791393 conn(0x7fa6c80fce20 msgr2=0x7fa6c8103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.836+0000 7fa6cce59640 1 -- 172.21.15.129:0/2498791393 shutdown_connections 2024-04-26T21:26:13.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.836+0000 7fa6cce59640 1 -- 172.21.15.129:0/2498791393 wait complete. 2024-04-26T21:26:13.847 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:13 smithi129 ceph-mon[29237]: pgmap v597: 97 pgs: 97 active+clean; 346 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 859 op/s 2024-04-26T21:26:13.848 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:26:13.848 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:26:13.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:13 smithi150 ceph-mon[32290]: pgmap v597: 97 pgs: 97 active+clean; 346 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 859 op/s 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.946+0000 7f016f79e640 1 -- 172.21.15.129:0/3577261912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0168108650 msgr2=0x7f016810aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.946+0000 7f016f79e640 1 --2- 172.21.15.129:0/3577261912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0168108650 0x7f016810aaa0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0160009ca0 tx=0x7f016002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.946+0000 7f016f79e640 1 -- 172.21.15.129:0/3577261912 shutdown_connections 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.946+0000 7f016f79e640 1 --2- 172.21.15.129:0/3577261912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0168108650 0x7f016810aaa0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.946+0000 7f016f79e640 1 --2- 172.21.15.129:0/3577261912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0168105d20 0x7f0168108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.946+0000 7f016f79e640 1 --2- 172.21.15.129:0/3577261912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01681033f0 0x7f01681057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.946+0000 7f016f79e640 1 -- 172.21.15.129:0/3577261912 >> 172.21.15.129:0/3577261912 conn(0x7f01680fcde0 msgr2=0x7f01680ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.946+0000 7f016f79e640 1 -- 172.21.15.129:0/3577261912 shutdown_connections 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.946+0000 7f016f79e640 1 -- 172.21.15.129:0/3577261912 wait complete. 2024-04-26T21:26:13.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016f79e640 1 Processor -- start 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016f79e640 1 -- start start 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016f79e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01681033f0 0x7f0168198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016f79e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168105d20 0x7f0168199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016f79e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0168108650 0x7f01681a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016f79e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f016810e4a0 con 0x7f01681033f0 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016f79e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f016810e320 con 0x7f0168105d20 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016f79e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f016810e620 con 0x7f0168108650 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016d53a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01681033f0 0x7f0168198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016d53a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01681033f0 0x7f0168198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51982/0 (socket says 172.21.15.129:51982) 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016d53a640 1 -- 172.21.15.129:0/700003724 learned_addr learned my addr 172.21.15.129:0/700003724 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016cd39640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168105d20 0x7f0168199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.947+0000 7f016dd3b640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0168108650 0x7f01681a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f016d53a640 1 -- 172.21.15.129:0/700003724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0168108650 msgr2=0x7f01681a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f016d53a640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0168108650 0x7f01681a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f016d53a640 1 -- 172.21.15.129:0/700003724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168105d20 msgr2=0x7f0168199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f016d53a640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168105d20 0x7f0168199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f016d53a640 1 -- 172.21.15.129:0/700003724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01681a0770 con 0x7f01681033f0 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f016d53a640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01681033f0 0x7f0168198b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f016400e9f0 tx=0x7f016400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f015e7fc640 1 -- 172.21.15.129:0/700003724 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016400cd80 con 0x7f01681033f0 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f015e7fc640 1 -- 172.21.15.129:0/700003724 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01640108e0 con 0x7f01681033f0 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01681a0a60 con 0x7f01681033f0 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f015e7fc640 1 -- 172.21.15.129:0/700003724 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164018980 con 0x7f01681033f0 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.948+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681a1000 con 0x7f01681033f0 2024-04-26T21:26:13.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.949+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01680fe8d0 con 0x7f01681033f0 2024-04-26T21:26:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.950+0000 7f015e7fc640 1 -- 172.21.15.129:0/700003724 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0164010430 con 0x7f01681033f0 2024-04-26T21:26:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.951+0000 7f015e7fc640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01540a3cc0 0x7f01540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.951+0000 7f015e7fc640 1 -- 172.21.15.129:0/700003724 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f01640d0ce0 con 0x7f01681033f0 2024-04-26T21:26:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.951+0000 7f016cd39640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01540a3cc0 0x7f01540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:13.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.951+0000 7f016cd39640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01540a3cc0 0x7f01540a6180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f016819a060 tx=0x7f015800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:13.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:13.955+0000 7f015e7fc640 1 -- 172.21.15.129:0/700003724 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f016409cf00 con 0x7f01681033f0 2024-04-26T21:26:13.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:13 smithi171 ceph-mon[33478]: pgmap v597: 97 pgs: 97 active+clean; 346 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 859 op/s 2024-04-26T21:26:14.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.182+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f016802dbf0 con 0x7f01681033f0 2024-04-26T21:26:14.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.183+0000 7f015e7fc640 1 -- 172.21.15.129:0/700003724 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f01640a1200 con 0x7f01681033f0 2024-04-26T21:26:14.185 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:26:14.185 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:26:14.185 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:26:14.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01540a3cc0 msgr2=0x7f01540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01540a3cc0 0x7f01540a6180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f016819a060 tx=0x7f015800b390 comp rx=0 tx=0).stop 2024-04-26T21:26:14.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01681033f0 msgr2=0x7f0168198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01681033f0 0x7f0168198b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f016400e9f0 tx=0x7f016400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 shutdown_connections 2024-04-26T21:26:14.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01540a3cc0 0x7f01540a6180 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168105d20 0x7f0168199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0168108650 0x7f01681a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 --2- 172.21.15.129:0/700003724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01681033f0 0x7f0168198b50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 >> 172.21.15.129:0/700003724 conn(0x7f01680fcde0 msgr2=0x7f0168103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:14.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 shutdown_connections 2024-04-26T21:26:14.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.185+0000 7f016f79e640 1 -- 172.21.15.129:0/700003724 wait complete. 2024-04-26T21:26:14.198 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:26:14.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.303+0000 7f53fac09640 1 -- 172.21.15.129:0/2480827851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4101870 msgr2=0x7f53f4101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.303+0000 7f53fac09640 1 --2- 172.21.15.129:0/2480827851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4101870 0x7f53f4101c90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009900 tx=0x7f53e802a410 comp rx=0 tx=0).stop 2024-04-26T21:26:14.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.303+0000 7f53fac09640 1 -- 172.21.15.129:0/2480827851 shutdown_connections 2024-04-26T21:26:14.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.303+0000 7f53fac09640 1 --2- 172.21.15.129:0/2480827851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f4103420 0x7f53f4109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.303+0000 7f53fac09640 1 --2- 172.21.15.129:0/2480827851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53f4102a60 0x7f53f4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.303+0000 7f53fac09640 1 --2- 172.21.15.129:0/2480827851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4101870 0x7f53f4101c90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.303+0000 7f53fac09640 1 -- 172.21.15.129:0/2480827851 >> 172.21.15.129:0/2480827851 conn(0x7f53f40fcde0 msgr2=0x7f53f40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:14.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 -- 172.21.15.129:0/2480827851 shutdown_connections 2024-04-26T21:26:14.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 -- 172.21.15.129:0/2480827851 wait complete. 2024-04-26T21:26:14.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 Processor -- start 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 -- start start 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53f4101870 0x7f53f419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4102a60 0x7f53f419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f4103420 0x7f53f41a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410e4a0 con 0x7f53f4102a60 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410e320 con 0x7f53f4103420 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.304+0000 7f53fac09640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410e620 con 0x7f53f4101870 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4102a60 0x7f53f419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4102a60 0x7f53f419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51996/0 (socket says 172.21.15.129:51996) 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f3fff640 1 -- 172.21.15.129:0/759368128 learned_addr learned my addr 172.21.15.129:0/759368128 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f91a6640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f4103420 0x7f53f41a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f89a5640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53f4101870 0x7f53f419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f3fff640 1 -- 172.21.15.129:0/759368128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53f4101870 msgr2=0x7f53f419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f3fff640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53f4101870 0x7f53f419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f3fff640 1 -- 172.21.15.129:0/759368128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f4103420 msgr2=0x7f53f41a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f3fff640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f4103420 0x7f53f41a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f3fff640 1 -- 172.21.15.129:0/759368128 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f41a29f0 con 0x7f53f4102a60 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f89a5640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53f4101870 0x7f53f419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f3fff640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4102a60 0x7f53f419b360 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c970 tx=0x7f53e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f1ffb640 1 -- 172.21.15.129:0/759368128 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0007bb0 con 0x7f53f4102a60 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53fac09640 1 -- 172.21.15.129:0/759368128 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f41a2ce0 con 0x7f53f4102a60 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f1ffb640 1 -- 172.21.15.129:0/759368128 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e0007d50 con 0x7f53f4102a60 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53f1ffb640 1 -- 172.21.15.129:0/759368128 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e001d060 con 0x7f53f4102a60 2024-04-26T21:26:14.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.305+0000 7f53fac09640 1 -- 172.21.15.129:0/759368128 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f41a3180 con 0x7f53f4102a60 2024-04-26T21:26:14.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53f1ffb640 1 -- 172.21.15.129:0/759368128 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f53e0004f00 con 0x7f53f4102a60 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53fda03740 1 Processor -- start 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53f1ffb640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f53e40a3c40 0x7f53e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53fda03740 1 -- start start 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53fda03740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb5ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53fda03740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 0x5585eb600e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53fda03740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb607c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53fda03740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5585eb563d80 con 0x5585eb5ff820 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53fda03740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5585eb51da20 con 0x5585eb6009f0 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53fda03740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5585eb55fd40 con 0x5585eb6013b0 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53f1ffb640 1 -- 172.21.15.129:0/759368128 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f53e00d02f0 con 0x7f53f4102a60 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53f91a6640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f53e40a3c40 0x7f53e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53f89a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb5ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53f3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 0x5585eb600e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53f89a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb5ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52006/0 (socket says 172.21.15.129:52006) 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53f89a5640 1 -- 172.21.15.129:0/260936224 learned_addr learned my addr 172.21.15.129:0/260936224 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.308+0000 7f53f91a6640 1 --2- 172.21.15.129:0/260936224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb607c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53f89a5640 1 -- 172.21.15.129:0/260936224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 msgr2=0x5585eb607c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53f89a5640 1 --2- 172.21.15.129:0/260936224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb607c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53f89a5640 1 -- 172.21.15.129:0/260936224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 msgr2=0x5585eb600e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53f89a5640 1 --2- 172.21.15.129:0/260936224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 0x5585eb600e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53f89a5640 1 -- 172.21.15.129:0/260936224 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585eb528c30 con 0x5585eb5ff820 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53f91a6640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f53e40a3c40 0x7f53e40a6100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009fd0 tx=0x7f53ec013b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53f89a5640 1 --2- 172.21.15.129:0/260936224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb5ffc40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f53e8006010 tx=0x7f53e8032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fd6f1c611e1b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53deffd640 1 -- 172.21.15.129:0/260936224 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e803f030 con 0x5585eb5ff820 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53deffd640 1 -- 172.21.15.129:0/260936224 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e8032d40 con 0x5585eb5ff820 2024-04-26T21:26:14.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.309+0000 7f53deffd640 1 -- 172.21.15.129:0/260936224 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8035070 con 0x5585eb5ff820 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 -- 172.21.15.129:0/260936224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 msgr2=0x5585eb5ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 --2- 172.21.15.129:0/260936224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb5ffc40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f53e8006010 tx=0x7f53e8032ba0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 -- 172.21.15.129:0/260936224 shutdown_connections 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 --2- 172.21.15.129:0/260936224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 0x5585eb600e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 --2- 172.21.15.129:0/260936224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb607c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 --2- 172.21.15.129:0/260936224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb5ffc40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 -- 172.21.15.129:0/260936224 >> 172.21.15.129:0/260936224 conn(0x5585eb540910 msgr2=0x5585eb5fb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 -- 172.21.15.129:0/260936224 shutdown_connections 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 -- 172.21.15.129:0/260936224 wait complete. 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 -- start start 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.310+0000 7f53fda03740 1 Processor -- start 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53fda03740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb6099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53fda03740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 0x5585eb609f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53fda03740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb610f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53fda03740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5585eb608360 con 0x5585eb5ff820 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53fda03740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5585eb4eadc0 con 0x5585eb6009f0 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53fda03740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5585eb6084e0 con 0x5585eb6013b0 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53fda03740 10 client.0 _ll_register_callbacks cb 0x5585eb506600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f89a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb6099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f89a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb6099e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52012/0 (socket says 172.21.15.129:52012) 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f89a5640 1 -- 172.21.15.129:0/1830235987 learned_addr learned my addr 172.21.15.129:0/1830235987 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f3fff640 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 0x5585eb609f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f89a5640 1 -- 172.21.15.129:0/1830235987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 msgr2=0x5585eb610f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f89a5640 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb610f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f89a5640 1 -- 172.21.15.129:0/1830235987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 msgr2=0x5585eb609f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f89a5640 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 0x5585eb609f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f89a5640 1 -- 172.21.15.129:0/1830235987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585eb2753f0 con 0x5585eb5ff820 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f3fff640 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 0x5585eb609f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53f89a5640 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb6099e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f53e8033000 tx=0x7f53e80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9869bab4a91cff34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53d3fff640 1 -- 172.21.15.129:0/1830235987 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8035050 con 0x5585eb5ff820 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53d3fff640 1 -- 172.21.15.129:0/1830235987 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f53e80318a0 con 0x5585eb5ff820 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.311+0000 7f53fda03740 1 -- 172.21.15.129:0/1830235987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e8032d40 con 0x5585eb5ff820 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53d3fff640 1 -- 172.21.15.129:0/1830235987 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f53e803f040 con 0x5585eb5ff820 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53fda03740 10 client.16113 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53fda03740 1 -- 172.21.15.129:0/1830235987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5585eb33f7d0 con 0x5585eb5ff820 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53d3fff640 1 -- 172.21.15.129:0/1830235987 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e8032d40 con 0x5585eb5ff820 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53d3fff640 1 -- 172.21.15.129:0/1830235987 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f53e80035f0 con 0x5585eb5ff820 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53fda03740 10 client.16113 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53fda03740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53fda03740 20 client.16113 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53fda03740 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5585eb615450 0x5585eb617840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53fda03740 4 client.16113 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53fda03740 1 -- 172.21.15.129:0/1830235987 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585eb571990 con 0x5585eb615450 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53f3fff640 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5585eb615450 0x5585eb617840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53d3fff640 10 client.16113 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.312+0000 7f53f3fff640 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5585eb615450 0x5585eb617840 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff16e9be7668ec2 server_cookie=b4d9b4402e2de2d8 in_seq=0 out_seq=0 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.313+0000 7f53d3fff640 1 -- 172.21.15.129:0/1830235987 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585eb571990 con 0x5585eb615450 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.313+0000 7f53d3fff640 10 client.16113 handle_command_reply: tid=0 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.313+0000 7f53fda03740 1 client.16113 shutdown 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.313+0000 7f53fda03740 1 -- 172.21.15.129:0/1830235987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 msgr2=0x5585eb6099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.313+0000 7f53fda03740 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb6099e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f53e8033000 tx=0x7f53e80043b0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53fda03740 1 -- 172.21.15.129:0/1830235987 shutdown_connections 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53fda03740 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5585eb615450 0x5585eb617840 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53fda03740 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb6009f0 0x5585eb609f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53fda03740 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb610f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53d3fff640 0 client.16113 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53fda03740 1 --2- 172.21.15.129:0/1830235987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb5ff820 0x5585eb6099e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53fda03740 1 -- 172.21.15.129:0/1830235987 >> 172.21.15.129:0/1830235987 conn(0x5585eb540910 msgr2=0x5585eb5fa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53fda03740 1 -- 172.21.15.129:0/1830235987 shutdown_connections 2024-04-26T21:26:14.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53fda03740 1 -- 172.21.15.129:0/1830235987 wait complete. 2024-04-26T21:26:14.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.314+0000 7f53fda03740 20 client.16113 trim_cache size 0 max 16384 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53fda03740 1 Processor -- start 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53fda03740 1 -- start start 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53fda03740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb608de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53fda03740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6009f0 0x5585eb610f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53fda03740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6013b0 0x5585eb609320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53fda03740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f53f41a9f60 con 0x5585eb6013b0 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53fda03740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f53f41ac7e0 con 0x5585eb5ff820 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53fda03740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f53f41aa5d0 con 0x5585eb6009f0 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53f91a6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6013b0 0x5585eb609320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53f91a6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6013b0 0x5585eb609320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52026/0 (socket says 172.21.15.129:52026) 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53f91a6640 1 -- 172.21.15.129:0/2301254513 learned_addr learned my addr 172.21.15.129:0/2301254513 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.327+0000 7f53f89a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb608de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53f3fff640 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6009f0 0x5585eb610f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53f89a5640 1 -- 172.21.15.129:0/2301254513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6009f0 msgr2=0x5585eb610f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53f89a5640 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6009f0 0x5585eb610f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53f89a5640 1 -- 172.21.15.129:0/2301254513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6013b0 msgr2=0x5585eb609320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53f89a5640 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6013b0 0x5585eb609320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53f89a5640 1 -- 172.21.15.129:0/2301254513 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585eb5f6ad0 con 0x5585eb5ff820 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53f3fff640 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6009f0 0x5585eb610f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53f91a6640 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6013b0 0x5585eb609320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53f89a5640 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb608de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f53e8003a90 tx=0x7f53e8004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d47734fe6cd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53de7fc640 1 -- 172.21.15.129:0/2301254513 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e805c070 con 0x5585eb5ff820 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53de7fc640 1 -- 172.21.15.129:0/2301254513 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e8004b70 con 0x5585eb5ff820 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.328+0000 7f53de7fc640 1 -- 172.21.15.129:0/2301254513 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8035070 con 0x5585eb5ff820 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 -- 172.21.15.129:0/2301254513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 msgr2=0x5585eb608de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb608de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f53e8003a90 tx=0x7f53e8004420 comp rx=0 tx=0).stop 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 -- 172.21.15.129:0/2301254513 shutdown_connections 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6013b0 0x5585eb609320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6009f0 0x5585eb610f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 --2- 172.21.15.129:0/2301254513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb608de0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 -- 172.21.15.129:0/2301254513 >> 172.21.15.129:0/2301254513 conn(0x5585eb540910 msgr2=0x5585eb5fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 -- 172.21.15.129:0/2301254513 shutdown_connections 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 -- 172.21.15.129:0/2301254513 wait complete. 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 -- start start 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 Processor -- start 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb60e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6009f0 0x5585eb60e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb60f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f53f41aaca0 con 0x5585eb6009f0 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f53f41ac110 con 0x5585eb5ff820 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f53f41ab370 con 0x5585eb6013b0 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.329+0000 7f53fda03740 10 client.0 _ll_register_callbacks cb 0x7f53e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f89a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb60e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6009f0 0x5585eb60e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6009f0 0x5585eb60e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52028/0 (socket says 172.21.15.129:52028) 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f89a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb60e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:32814/0 (socket says 172.21.15.129:32814) 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f3fff640 1 -- 172.21.15.129:0/3084818900 learned_addr learned my addr 172.21.15.129:0/3084818900 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f91a6640 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb60f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f3fff640 1 -- 172.21.15.129:0/3084818900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 msgr2=0x5585eb60f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f3fff640 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb60f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f3fff640 1 -- 172.21.15.129:0/3084818900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 msgr2=0x5585eb60e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f3fff640 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb60e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f3fff640 1 -- 172.21.15.129:0/3084818900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585eb60fda0 con 0x5585eb6009f0 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53f3fff640 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6009f0 0x5585eb60e7f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f53e0098de0 tx=0x7f53e0098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b006cbcc5c5ec0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53d3fff640 1 -- 172.21.15.129:0/3084818900 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00a50d0 con 0x5585eb6009f0 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53d3fff640 1 -- 172.21.15.129:0/3084818900 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f53e00046c0 con 0x5585eb6009f0 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53fda03740 1 -- 172.21.15.129:0/3084818900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e8004b70 con 0x5585eb6009f0 2024-04-26T21:26:14.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53d3fff640 1 -- 172.21.15.129:0/3084818900 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f53e00cdee0 con 0x5585eb6009f0 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53fda03740 10 client.16119 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53fda03740 1 -- 172.21.15.129:0/3084818900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5585eb611110 con 0x5585eb6009f0 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.330+0000 7f53d3fff640 1 -- 172.21.15.129:0/3084818900 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e8004b70 con 0x5585eb6009f0 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53d3fff640 1 -- 172.21.15.129:0/3084818900 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f53e00a4c40 con 0x5585eb6009f0 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53fda03740 10 client.16119 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53fda03740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53fda03740 20 client.16119 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53fda03740 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5585eb67bff0 0x5585eb67e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53fda03740 4 client.16119 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53fda03740 1 -- 172.21.15.129:0/3084818900 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585eb56b800 con 0x5585eb67bff0 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53f89a5640 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5585eb67bff0 0x5585eb67e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53f89a5640 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5585eb67bff0 0x5585eb67e410 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b466afc831a52fb2 server_cookie=a7461eb6bbfefd06 in_seq=0 out_seq=0 2024-04-26T21:26:14.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.331+0000 7f53d3fff640 10 client.16119 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53d3fff640 1 -- 172.21.15.129:0/3084818900 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585eb56b800 con 0x5585eb67bff0 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53d3fff640 10 client.16119 handle_command_reply: tid=0 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 client.16119 shutdown 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 -- 172.21.15.129:0/3084818900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6009f0 msgr2=0x5585eb60e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6009f0 0x5585eb60e7f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f53e0098de0 tx=0x7f53e0098040 comp rx=0 tx=0).stop 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 -- 172.21.15.129:0/3084818900 shutdown_connections 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5585eb67bff0 0x5585eb67e410 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585eb6013b0 0x5585eb60f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585eb6009f0 0x5585eb60e7f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 --2- 172.21.15.129:0/3084818900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585eb5ff820 0x5585eb60e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53d3fff640 0 client.16119 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:14.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 -- 172.21.15.129:0/3084818900 >> 172.21.15.129:0/3084818900 conn(0x5585eb540910 msgr2=0x5585eb5fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:14.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 -- 172.21.15.129:0/3084818900 shutdown_connections 2024-04-26T21:26:14.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 1 -- 172.21.15.129:0/3084818900 wait complete. 2024-04-26T21:26:14.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fda03740 20 client.16119 trim_cache size 0 max 16384 2024-04-26T21:26:14.341 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:26:14.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fac09640 1 -- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f53e40a3c40 msgr2=0x7f53e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fac09640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f53e40a3c40 0x7f53e40a6100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009fd0 tx=0x7f53ec013b80 comp rx=0 tx=0).stop 2024-04-26T21:26:14.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fac09640 1 -- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4102a60 msgr2=0x7f53f419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fac09640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4102a60 0x7f53f419b360 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c970 tx=0x7f53e000ce30 comp rx=0 tx=0).stop 2024-04-26T21:26:14.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fac09640 1 -- 172.21.15.129:0/759368128 shutdown_connections 2024-04-26T21:26:14.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fac09640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f53e40a3c40 0x7f53e40a6100 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fac09640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f4103420 0x7f53f41a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fac09640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53f4102a60 0x7f53f419b360 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.333+0000 7f53fac09640 1 --2- 172.21.15.129:0/759368128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53f4101870 0x7f53f419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.334+0000 7f53fac09640 1 -- 172.21.15.129:0/759368128 >> 172.21.15.129:0/759368128 conn(0x7f53f40fcde0 msgr2=0x7f53f4108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:14.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.334+0000 7f53fac09640 1 -- 172.21.15.129:0/759368128 shutdown_connections 2024-04-26T21:26:14.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:14.334+0000 7f53fac09640 1 -- 172.21.15.129:0/759368128 wait complete. 2024-04-26T21:26:14.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:14 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/700003724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:26:14.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:14 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/700003724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:26:14.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:14 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/700003724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:26:15.347 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:26:15.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:26:15.347 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:26:15.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.464+0000 7f602e259640 1 -- 172.21.15.129:0/480557869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028102970 msgr2=0x7f6028102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.464+0000 7f602e259640 1 --2- 172.21.15.129:0/480557869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028102970 0x7f6028102d90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f601c009900 tx=0x7f601c02a410 comp rx=0 tx=0).stop 2024-04-26T21:26:15.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.464+0000 7f602e259640 1 -- 172.21.15.129:0/480557869 shutdown_connections 2024-04-26T21:26:15.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.464+0000 7f602e259640 1 --2- 172.21.15.129:0/480557869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6028104520 0x7f602810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.464+0000 7f602e259640 1 --2- 172.21.15.129:0/480557869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6028103b60 0x7f6028103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.464+0000 7f602e259640 1 --2- 172.21.15.129:0/480557869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028102970 0x7f6028102d90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.464+0000 7f602e259640 1 -- 172.21.15.129:0/480557869 >> 172.21.15.129:0/480557869 conn(0x7f60280fdee0 msgr2=0x7f6028100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.464+0000 7f602e259640 1 -- 172.21.15.129:0/480557869 shutdown_connections 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.464+0000 7f602e259640 1 -- 172.21.15.129:0/480557869 wait complete. 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f602e259640 1 Processor -- start 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f602e259640 1 -- start start 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f602e259640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6028102970 0x7f6028198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f602e259640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028103b60 0x7f60281990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f602e259640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6028104520 0x7f60281a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f602e259640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f602810e470 con 0x7f6028103b60 2024-04-26T21:26:15.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f602e259640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f602810e2f0 con 0x7f6028102970 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f602e259640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f602810e5f0 con 0x7f6028104520 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6026ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028103b60 0x7f60281990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6026ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028103b60 0x7f60281990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52042/0 (socket says 172.21.15.129:52042) 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6026ffd640 1 -- 172.21.15.129:0/1467801324 learned_addr learned my addr 172.21.15.129:0/1467801324 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f60277fe640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6028102970 0x7f6028198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6027fff640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6028104520 0x7f60281a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6026ffd640 1 -- 172.21.15.129:0/1467801324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6028104520 msgr2=0x7f60281a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6026ffd640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6028104520 0x7f60281a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6026ffd640 1 -- 172.21.15.129:0/1467801324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6028102970 msgr2=0x7f6028198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6026ffd640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6028102970 0x7f6028198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6026ffd640 1 -- 172.21.15.129:0/1467801324 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60281a0790 con 0x7f6028103b60 2024-04-26T21:26:15.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f6027fff640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6028104520 0x7f60281a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:26:15.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.465+0000 7f60277fe640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6028102970 0x7f6028198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:15.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.466+0000 7f6026ffd640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028103b60 0x7f60281990b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f601800c920 tx=0x7f601800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.466+0000 7f6024ff9640 1 -- 172.21.15.129:0/1467801324 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018007c70 con 0x7f6028103b60 2024-04-26T21:26:15.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.466+0000 7f6024ff9640 1 -- 172.21.15.129:0/1467801324 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6018007e10 con 0x7f6028103b60 2024-04-26T21:26:15.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.466+0000 7f602e259640 1 -- 172.21.15.129:0/1467801324 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60281a0a80 con 0x7f6028103b60 2024-04-26T21:26:15.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.466+0000 7f6024ff9640 1 -- 172.21.15.129:0/1467801324 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018018820 con 0x7f6028103b60 2024-04-26T21:26:15.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.466+0000 7f602e259640 1 -- 172.21.15.129:0/1467801324 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281a0fd0 con 0x7f6028103b60 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.468+0000 7f6024ff9640 1 -- 172.21.15.129:0/1467801324 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f601800f450 con 0x7f6028103b60 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.468+0000 7f6030f73740 1 Processor -- start 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.468+0000 7f6030f73740 1 -- start start 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.468+0000 7f6030f73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x56018277fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.468+0000 7f6030f73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x560182784da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.468+0000 7f6030f73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560182785410 0x5601827878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6030f73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5601826e4b00 con 0x56018277f8b0 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6030f73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560182680920 con 0x5601827802a0 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6030f73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5601826e0ac0 con 0x560182785410 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x56018277fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x56018277fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52046/0 (socket says 172.21.15.129:52046) 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 -- 172.21.15.129:0/2274230590 learned_addr learned my addr 172.21.15.129:0/2274230590 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f60277fe640 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x560182784da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6024ff9640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60100a3c40 0x7f60100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 -- 172.21.15.129:0/2274230590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560182785410 msgr2=0x5601827878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6026ffd640 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560182785410 0x5601827878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560182785410 0x5601827878a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 -- 172.21.15.129:0/2274230590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 msgr2=0x560182784da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x560182784da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 -- 172.21.15.129:0/2274230590 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601824be7e0 con 0x56018277f8b0 2024-04-26T21:26:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f60277fe640 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x560182784da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6024ff9640 1 -- 172.21.15.129:0/1467801324 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f60180d06b0 con 0x7f6028103b60 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60100a3c40 0x7f60100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f6027fff640 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x56018277fcd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6020009fd0 tx=0x7f60200108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c812b752693b9310 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f600dffb640 1 -- 172.21.15.129:0/2274230590 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020022070 con 0x56018277f8b0 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f600dffb640 1 -- 172.21.15.129:0/2274230590 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602001b380 con 0x56018277f8b0 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.469+0000 7f600dffb640 1 -- 172.21.15.129:0/2274230590 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602000e5c0 con 0x56018277f8b0 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 -- 172.21.15.129:0/2274230590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 msgr2=0x56018277fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x56018277fcd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6020009fd0 tx=0x7f60200108e0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6027fff640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60100a3c40 0x7f60100a6100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f602001f000 tx=0x7f602001c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 -- 172.21.15.129:0/2274230590 shutdown_connections 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x560182784da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560182785410 0x5601827878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 --2- 172.21.15.129:0/2274230590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x56018277fcd0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 -- 172.21.15.129:0/2274230590 >> 172.21.15.129:0/2274230590 conn(0x56018277af20 msgr2=0x56018277b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 -- 172.21.15.129:0/2274230590 shutdown_connections 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 -- 172.21.15.129:0/2274230590 wait complete. 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.470+0000 7f6030f73740 1 -- start start 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6030f73740 1 Processor -- start 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6030f73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 0x560182789ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6030f73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6030f73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x560182791220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6030f73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5601827883c0 con 0x560182785410 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6030f73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56018267e5d0 con 0x5601827802a0 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6030f73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560182788540 con 0x56018277f8b0 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6030f73740 10 client.0 _ll_register_callbacks cb 0x56018266c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6027fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x560182791220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6027fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x560182791220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52060/0 (socket says 172.21.15.129:52060) 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6027fff640 1 -- 172.21.15.129:0/2987347258 learned_addr learned my addr 172.21.15.129:0/2987347258 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6026ffd640 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f60277fe640 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 0x560182789ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6027fff640 1 -- 172.21.15.129:0/2987347258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 msgr2=0x560182789ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6027fff640 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 0x560182789ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6027fff640 1 -- 172.21.15.129:0/2987347258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 msgr2=0x56018278a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6027fff640 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6027fff640 1 -- 172.21.15.129:0/2987347258 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018251ef70 con 0x560182785410 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6026ffd640 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6027fff640 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x560182791220 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6020020040 tx=0x7f602001cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991c10961e9c379a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.471+0000 7f6006ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6006ffd640 1 -- 172.21.15.129:0/2987347258 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020022070 con 0x560182785410 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6006ffd640 1 -- 172.21.15.129:0/2987347258 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f60200049c0 con 0x560182785410 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6030f73740 1 -- 172.21.15.129:0/2987347258 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602001b380 con 0x560182785410 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6006ffd640 1 -- 172.21.15.129:0/2987347258 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6020009c90 con 0x560182785410 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6030f73740 10 client.16131 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6030f73740 1 -- 172.21.15.129:0/2987347258 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5601823b6170 con 0x560182785410 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6006ffd640 1 -- 172.21.15.129:0/2987347258 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602001b380 con 0x560182785410 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6006ffd640 1 -- 172.21.15.129:0/2987347258 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f602002f020 con 0x560182785410 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6030f73740 10 client.16131 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6030f73740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6030f73740 20 client.16131 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6030f73740 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x560182795490 0x5601827978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6030f73740 4 client.16131 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f6030f73740 1 -- 172.21.15.129:0/2987347258 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601826f2710 con 0x560182795490 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.472+0000 7f60277fe640 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x560182795490 0x5601827978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.473+0000 7f6006ffd640 10 client.16131 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.473+0000 7f60277fe640 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x560182795490 0x5601827978f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a73ba15ddb2cbd server_cookie=ccb794451df6a14 in_seq=0 out_seq=0 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.473+0000 7f6006ffd640 1 -- 172.21.15.129:0/2987347258 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601826f2710 con 0x560182795490 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.473+0000 7f6006ffd640 10 client.16131 handle_command_reply: tid=0 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 client.16131 shutdown 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 -- 172.21.15.129:0/2987347258 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 msgr2=0x560182791220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x560182791220 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6020020040 tx=0x7f602001cd20 comp rx=0 tx=0).stop 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 -- 172.21.15.129:0/2987347258 shutdown_connections 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x560182795490 0x5601827978f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x560182791220 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 --2- 172.21.15.129:0/2987347258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 0x560182789ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6006ffd640 0 client.16131 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:15.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 -- 172.21.15.129:0/2987347258 >> 172.21.15.129:0/2987347258 conn(0x56018277af20 msgr2=0x56018277a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 -- 172.21.15.129:0/2987347258 shutdown_connections 2024-04-26T21:26:15.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 1 -- 172.21.15.129:0/2987347258 wait complete. 2024-04-26T21:26:15.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.474+0000 7f6030f73740 20 client.16131 trim_cache size 0 max 16384 2024-04-26T21:26:15.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f6030f73740 1 Processor -- start 2024-04-26T21:26:15.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f6030f73740 1 -- start start 2024-04-26T21:26:15.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f6030f73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x560182779850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f6030f73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601827802a0 0x5601827911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f6030f73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560182785410 0x560182779d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f6030f73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f60281a7dc0 con 0x56018277f8b0 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f6030f73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60281aa640 con 0x560182785410 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f6030f73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f60281a8430 con 0x5601827802a0 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f60277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x560182779850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f60277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x560182779850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52064/0 (socket says 172.21.15.129:52064) 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.491+0000 7f60277fe640 1 -- 172.21.15.129:0/227865361 learned_addr learned my addr 172.21.15.129:0/227865361 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f6027fff640 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560182785410 0x560182779d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f6026ffd640 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601827802a0 0x5601827911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f60277fe640 1 -- 172.21.15.129:0/227865361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601827802a0 msgr2=0x5601827911e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f60277fe640 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601827802a0 0x5601827911e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f60277fe640 1 -- 172.21.15.129:0/227865361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560182785410 msgr2=0x560182779d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f60277fe640 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560182785410 0x560182779d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f60277fe640 1 -- 172.21.15.129:0/227865361 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601827882a0 con 0x56018277f8b0 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f6026ffd640 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601827802a0 0x5601827911e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f60277fe640 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x560182779850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f601c009230 tx=0x7f601c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7d7eb892221e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f600d7fa640 1 -- 172.21.15.129:0/227865361 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c035040 con 0x56018277f8b0 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f600d7fa640 1 -- 172.21.15.129:0/227865361 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f601c04a070 con 0x56018277f8b0 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.492+0000 7f600d7fa640 1 -- 172.21.15.129:0/227865361 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c0452c0 con 0x56018277f8b0 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 -- 172.21.15.129:0/227865361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 msgr2=0x560182779850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x560182779850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f601c009230 tx=0x7f601c0314a0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 -- 172.21.15.129:0/227865361 shutdown_connections 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560182785410 0x560182779d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601827802a0 0x5601827911e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 --2- 172.21.15.129:0/227865361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018277f8b0 0x560182779850 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 -- 172.21.15.129:0/227865361 >> 172.21.15.129:0/227865361 conn(0x56018277af20 msgr2=0x560182781670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 -- 172.21.15.129:0/227865361 shutdown_connections 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 -- 172.21.15.129:0/227865361 wait complete. 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 -- start start 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 Processor -- start 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 0x56018278e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x56018278fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f60281a8b00 con 0x560182785410 2024-04-26T21:26:15.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60281a9f70 con 0x5601827802a0 2024-04-26T21:26:15.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f60281a91d0 con 0x56018277f8b0 2024-04-26T21:26:15.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6030f73740 10 client.0 _ll_register_callbacks cb 0x7f6020009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:15.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.493+0000 7f6027fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x56018278fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6027fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x56018278fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52078/0 (socket says 172.21.15.129:52078) 2024-04-26T21:26:15.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6027fff640 1 -- 172.21.15.129:0/1583696077 learned_addr learned my addr 172.21.15.129:0/1583696077 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6026ffd640 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f60277fe640 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 0x56018278e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6027fff640 1 -- 172.21.15.129:0/1583696077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 msgr2=0x56018278e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6027fff640 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 0x56018278e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6027fff640 1 -- 172.21.15.129:0/1583696077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 msgr2=0x56018278ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6027fff640 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6027fff640 1 -- 172.21.15.129:0/1583696077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560182790460 con 0x560182785410 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6026ffd640 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6027fff640 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x56018278fe70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6020028480 tx=0x7f6020005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e4a000c9f934e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6006ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6006ffd640 1 -- 172.21.15.129:0/1583696077 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020028700 con 0x560182785410 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6006ffd640 1 -- 172.21.15.129:0/1583696077 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f60200288a0 con 0x560182785410 2024-04-26T21:26:15.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6030f73740 1 -- 172.21.15.129:0/1583696077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601c04a070 con 0x560182785410 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6006ffd640 1 -- 172.21.15.129:0/1583696077 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6020009c90 con 0x560182785410 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6030f73740 10 client.16137 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6030f73740 1 -- 172.21.15.129:0/1583696077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5601827fba30 con 0x560182785410 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.494+0000 7f6006ffd640 1 -- 172.21.15.129:0/1583696077 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f601c04a070 con 0x560182785410 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f6006ffd640 1 -- 172.21.15.129:0/1583696077 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f6020048050 con 0x560182785410 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f6030f73740 10 client.16137 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f6030f73740 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f6030f73740 20 client.16137 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f6030f73740 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5601827fc120 0x5601827fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f6030f73740 4 client.16137 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f6030f73740 1 -- 172.21.15.129:0/1583696077 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560182789a90 con 0x5601827fc120 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f60277fe640 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5601827fc120 0x5601827fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f6006ffd640 10 client.16137 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.495+0000 7f60277fe640 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5601827fc120 0x5601827fe540 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4653aa8e247745c8 server_cookie=67f21e9bf1a85ff6 in_seq=0 out_seq=0 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6006ffd640 1 -- 172.21.15.129:0/1583696077 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560182789a90 con 0x5601827fc120 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6006ffd640 10 client.16137 handle_command_reply: tid=0 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 client.16137 shutdown 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 -- 172.21.15.129:0/1583696077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 msgr2=0x56018278fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x56018278fe70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6020028480 tx=0x7f6020005b00 comp rx=0 tx=0).stop 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 -- 172.21.15.129:0/1583696077 shutdown_connections 2024-04-26T21:26:15.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5601827fc120 0x5601827fe540 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601827802a0 0x56018278ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560182785410 0x56018278fe70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6006ffd640 0 client.16137 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 --2- 172.21.15.129:0/1583696077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56018277f8b0 0x56018278e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 -- 172.21.15.129:0/1583696077 >> 172.21.15.129:0/1583696077 conn(0x56018277af20 msgr2=0x560182779850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 -- 172.21.15.129:0/1583696077 shutdown_connections 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 1 -- 172.21.15.129:0/1583696077 wait complete. 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f6030f73740 20 client.16137 trim_cache size 0 max 16384 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:26:15.509 INFO:teuthology.orchestra.run.smithi129.stdout: "return_code": 0, 2024-04-26T21:26:15.510 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_tag": "cbe4dae2-bfb3-4718-ab3a-199872b33243", 2024-04-26T21:26:15.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": "asynchronous" 2024-04-26T21:26:15.510 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 -- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60100a3c40 msgr2=0x7f60100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60100a3c40 0x7f60100a6100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f602001f000 tx=0x7f602001c000 comp rx=0 tx=0).stop 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 -- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028103b60 msgr2=0x7f60281990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028103b60 0x7f60281990b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f601800c920 tx=0x7f601800cde0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 -- 172.21.15.129:0/1467801324 shutdown_connections 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60100a3c40 0x7f60100a6100 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6028104520 0x7f60281a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6028103b60 0x7f60281990b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 --2- 172.21.15.129:0/1467801324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6028102970 0x7f6028198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.497+0000 7f602e259640 1 -- 172.21.15.129:0/1467801324 >> 172.21.15.129:0/1467801324 conn(0x7f60280fdee0 msgr2=0x7f6028106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.498+0000 7f602e259640 1 -- 172.21.15.129:0/1467801324 shutdown_connections 2024-04-26T21:26:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.498+0000 7f602e259640 1 -- 172.21.15.129:0/1467801324 wait complete. 2024-04-26T21:26:15.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:26:15.513 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:26:15.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.615+0000 7f08541ac640 1 -- 172.21.15.129:0/1835891739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c105d20 msgr2=0x7f084c108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.615+0000 7f08541ac640 1 --2- 172.21.15.129:0/1835891739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c105d20 0x7f084c108110 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0848009900 tx=0x7f084802a410 comp rx=0 tx=0).stop 2024-04-26T21:26:15.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 -- 172.21.15.129:0/1835891739 shutdown_connections 2024-04-26T21:26:15.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 --2- 172.21.15.129:0/1835891739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f084c108650 0x7f084c10aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 --2- 172.21.15.129:0/1835891739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c105d20 0x7f084c108110 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 --2- 172.21.15.129:0/1835891739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f084c1033f0 0x7f084c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 -- 172.21.15.129:0/1835891739 >> 172.21.15.129:0/1835891739 conn(0x7f084c0fce20 msgr2=0x7f084c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 -- 172.21.15.129:0/1835891739 shutdown_connections 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 -- 172.21.15.129:0/1835891739 wait complete. 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 Processor -- start 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 -- start start 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f084c1033f0 0x7f084c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f084c105d20 0x7f084c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c108650 0x7f084c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f084c10e490 con 0x7f084c108650 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f084c10e310 con 0x7f084c1033f0 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f08541ac640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f084c10e610 con 0x7f084c105d20 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f0852749640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c108650 0x7f084c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f0852749640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c108650 0x7f084c1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52092/0 (socket says 172.21.15.129:52092) 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f0851f48640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f084c1033f0 0x7f084c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f0852749640 1 -- 172.21.15.129:0/3116179615 learned_addr learned my addr 172.21.15.129:0/3116179615 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.616+0000 7f0851747640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f084c105d20 0x7f084c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0852749640 1 -- 172.21.15.129:0/3116179615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f084c105d20 msgr2=0x7f084c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0852749640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f084c105d20 0x7f084c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0852749640 1 -- 172.21.15.129:0/3116179615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f084c1033f0 msgr2=0x7f084c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0852749640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f084c1033f0 0x7f084c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0852749640 1 -- 172.21.15.129:0/3116179615 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c1a0800 con 0x7f084c108650 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0851f48640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f084c1033f0 0x7f084c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0852749640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c108650 0x7f084c1a0190 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f084400eaa0 tx=0x7f084400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0842ffd640 1 -- 172.21.15.129:0/3116179615 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084400cdb0 con 0x7f084c108650 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f08541ac640 1 -- 172.21.15.129:0/3116179615 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c1a0af0 con 0x7f084c108650 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0842ffd640 1 -- 172.21.15.129:0/3116179615 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0844005020 con 0x7f084c108650 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f0842ffd640 1 -- 172.21.15.129:0/3116179615 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08440116d0 con 0x7f084c108650 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.617+0000 7f08541ac640 1 -- 172.21.15.129:0/3116179615 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c1a0fc0 con 0x7f084c108650 2024-04-26T21:26:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.619+0000 7f0842ffd640 1 -- 172.21.15.129:0/3116179615 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f08440051d0 con 0x7f084c108650 2024-04-26T21:26:15.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0856f9c740 1 Processor -- start 2024-04-26T21:26:15.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0856f9c740 1 -- start start 2024-04-26T21:26:15.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0842ffd640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f08380a3c40 0x7f08380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f90300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 0x55e5c6f91530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6f98300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0856f9c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e5c6ef4d80 con 0x55e5c6f91a70 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0856f9c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e5c6eae8b0 con 0x55e5c6f910b0 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0856f9c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e5c6ef0d40 con 0x55e5c6f8fee0 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0842ffd640 1 -- 172.21.15.129:0/3116179615 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f08440d06f0 con 0x7f084c108650 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0851747640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f08380a3c40 0x7f08380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0851747640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 0x55e5c6f91530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0852749640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6f98300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0852749640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6f98300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52106/0 (socket says 172.21.15.129:52106) 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0851747640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 0x55e5c6f91530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:32890/0 (socket says 172.21.15.129:32890) 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0852749640 1 -- 172.21.15.129:0/1714743531 learned_addr learned my addr 172.21.15.129:0/1714743531 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.620+0000 7f0851f48640 1 --2- 172.21.15.129:0/1714743531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f90300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0852749640 1 -- 172.21.15.129:0/1714743531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 msgr2=0x55e5c6f90300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0852749640 1 --2- 172.21.15.129:0/1714743531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f90300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0852749640 1 -- 172.21.15.129:0/1714743531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 msgr2=0x55e5c6f91530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0852749640 1 --2- 172.21.15.129:0/1714743531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 0x55e5c6f91530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0852749640 1 -- 172.21.15.129:0/1714743531 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c6e7fa40 con 0x55e5c6f91a70 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0851f48640 1 --2- 172.21.15.129:0/1714743531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f90300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0851747640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f08380a3c40 0x7f08380a6100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0848009230 tx=0x7f08480095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0852749640 1 --2- 172.21.15.129:0/1714743531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6f98300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f08440a4fa0 tx=0x7f084409c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad326e075ee46384 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0837fff640 1 -- 172.21.15.129:0/1714743531 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084401f040 con 0x55e5c6f91a70 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0837fff640 1 -- 172.21.15.129:0/1714743531 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0844098f80 con 0x55e5c6f91a70 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.621+0000 7f0837fff640 1 -- 172.21.15.129:0/1714743531 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084409db90 con 0x55e5c6f91a70 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 -- 172.21.15.129:0/1714743531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 msgr2=0x55e5c6f98300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1714743531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6f98300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f08440a4fa0 tx=0x7f084409c3a0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 -- 172.21.15.129:0/1714743531 shutdown_connections 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1714743531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 0x55e5c6f91530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1714743531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6f98300 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1714743531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f90300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 -- 172.21.15.129:0/1714743531 >> 172.21.15.129:0/1714743531 conn(0x55e5c6f8b3d0 msgr2=0x55e5c6f8b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 -- 172.21.15.129:0/1714743531 shutdown_connections 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 -- 172.21.15.129:0/1714743531 wait complete. 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 -- start start 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 Processor -- start 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f9a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e5c6f98a20 con 0x55e5c6f910b0 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e5c6e7bd10 con 0x55e5c6f8fee0 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.622+0000 7f0856f9c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e5c6f98ba0 con 0x55e5c6f91a70 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0856f9c740 10 client.0 _ll_register_callbacks cb 0x55e5c6ea8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851747640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851f48640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f9a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851747640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52120/0 (socket says 172.21.15.129:52120) 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851747640 1 -- 172.21.15.129:0/1432480537 learned_addr learned my addr 172.21.15.129:0/1432480537 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0852749640 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851747640 1 -- 172.21.15.129:0/1432480537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 msgr2=0x55e5c6fa18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851747640 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851747640 1 -- 172.21.15.129:0/1432480537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 msgr2=0x55e5c6f9a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851747640 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f9a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851747640 1 -- 172.21.15.129:0/1432480537 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c6cd07d0 con 0x55e5c6f910b0 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851f48640 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f9a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0851747640 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9a960 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0848000c00 tx=0x7f084802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4925953b61d363 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f08357fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f08357fa640 1 -- 172.21.15.129:0/1432480537 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848035050 con 0x55e5c6f910b0 2024-04-26T21:26:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f08357fa640 1 -- 172.21.15.129:0/1432480537 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0848004410 con 0x55e5c6f910b0 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0856f9c740 1 -- 172.21.15.129:0/1432480537 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0844098f80 con 0x55e5c6f910b0 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f08357fa640 1 -- 172.21.15.129:0/1432480537 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f08480035f0 con 0x55e5c6f910b0 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0856f9c740 10 client.16149 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f0856f9c740 1 -- 172.21.15.129:0/1432480537 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e5c6c063f0 con 0x55e5c6f910b0 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.623+0000 7f08357fa640 1 -- 172.21.15.129:0/1432480537 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0844098f80 con 0x55e5c6f910b0 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f08357fa640 1 -- 172.21.15.129:0/1432480537 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f084804a070 con 0x55e5c6f910b0 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f0856f9c740 10 client.16149 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f0856f9c740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f0856f9c740 20 client.16149 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5c6fa5e40 0x55e5c6fa82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f0856f9c740 4 client.16149 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f0856f9c740 1 -- 172.21.15.129:0/1432480537 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5c6f02990 con 0x55e5c6fa5e40 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f0851f48640 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5c6fa5e40 0x55e5c6fa82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f0851f48640 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5c6fa5e40 0x55e5c6fa82e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c3e6f811d1debbf server_cookie=2aa6b1d6213098e8 in_seq=0 out_seq=0 2024-04-26T21:26:15.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.624+0000 7f08357fa640 10 client.16149 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:15.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f08357fa640 1 -- 172.21.15.129:0/1432480537 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5c6f02990 con 0x55e5c6fa5e40 2024-04-26T21:26:15.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f08357fa640 10 client.16149 handle_command_reply: tid=0 2024-04-26T21:26:15.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 client.16149 shutdown 2024-04-26T21:26:15.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 -- 172.21.15.129:0/1432480537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 msgr2=0x55e5c6f9a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9a960 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0848000c00 tx=0x7f084802a410 comp rx=0 tx=0).stop 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 -- 172.21.15.129:0/1432480537 shutdown_connections 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5c6fa5e40 0x55e5c6fa82e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9a960 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1432480537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f9a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f08357fa640 0 client.16149 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 -- 172.21.15.129:0/1432480537 >> 172.21.15.129:0/1432480537 conn(0x55e5c6f8b3d0 msgr2=0x55e5c6f8a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 -- 172.21.15.129:0/1432480537 shutdown_connections 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 1 -- 172.21.15.129:0/1432480537 wait complete. 2024-04-26T21:26:15.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.639+0000 7f0856f9c740 20 client.16149 trim_cache size 0 max 16384 2024-04-26T21:26:15.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.661+0000 7f0856f9c740 1 Processor -- start 2024-04-26T21:26:15.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0856f9c740 1 -- start start 2024-04-26T21:26:15.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f8e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 0x55e5c6fa1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 0x55e5c6f8e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0856f9c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1a7cd0 con 0x55e5c6f910b0 2024-04-26T21:26:15.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0856f9c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1aa550 con 0x55e5c6f8fee0 2024-04-26T21:26:15.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0856f9c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1a8340 con 0x55e5c6f91a70 2024-04-26T21:26:15.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0852749640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 0x55e5c6f8e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0851f48640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f8e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0851f48640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f8e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.129:32900/0 (socket says 172.21.15.129:32900) 2024-04-26T21:26:15.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.662+0000 7f0851f48640 1 -- 172.21.15.129:0/256677352 learned_addr learned my addr 172.21.15.129:0/256677352 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.663+0000 7f0851f48640 1 -- 172.21.15.129:0/256677352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 msgr2=0x55e5c6f8e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.663+0000 7f0851f48640 1 --2- 172.21.15.129:0/256677352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 0x55e5c6f8e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.663+0000 7f0851f48640 1 -- 172.21.15.129:0/256677352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 msgr2=0x55e5c6fa1880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:26:15.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.663+0000 7f0851f48640 1 --2- 172.21.15.129:0/256677352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 0x55e5c6fa1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.663+0000 7f0851f48640 1 -- 172.21.15.129:0/256677352 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c6bc60f0 con 0x55e5c6f8fee0 2024-04-26T21:26:15.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.664+0000 7f0851f48640 1 --2- 172.21.15.129:0/256677352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f8e2e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f083c009460 tx=0x7f083c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e8d9c954c164c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.664+0000 7f08377fe640 1 -- 172.21.15.129:0/256677352 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c004030 con 0x55e5c6f8fee0 2024-04-26T21:26:15.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.664+0000 7f08377fe640 1 -- 172.21.15.129:0/256677352 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083c00f8f0 con 0x55e5c6f8fee0 2024-04-26T21:26:15.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.664+0000 7f08377fe640 1 -- 172.21.15.129:0/256677352 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c005280 con 0x55e5c6f8fee0 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.665+0000 7f0856f9c740 1 -- 172.21.15.129:0/256677352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 msgr2=0x55e5c6f8e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.665+0000 7f0856f9c740 1 --2- 172.21.15.129:0/256677352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f8e2e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f083c009460 tx=0x7f083c005f90 comp rx=0 tx=0).stop 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.665+0000 7f0856f9c740 1 -- 172.21.15.129:0/256677352 shutdown_connections 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.665+0000 7f0856f9c740 1 --2- 172.21.15.129:0/256677352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f91a70 0x55e5c6f8e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.665+0000 7f0856f9c740 1 --2- 172.21.15.129:0/256677352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f910b0 0x55e5c6fa1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.665+0000 7f0856f9c740 1 --2- 172.21.15.129:0/256677352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f8e2e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.665+0000 7f0856f9c740 1 -- 172.21.15.129:0/256677352 >> 172.21.15.129:0/256677352 conn(0x55e5c6f8b3d0 msgr2=0x55e5c6f940f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.665+0000 7f0856f9c740 1 -- 172.21.15.129:0/256677352 shutdown_connections 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.665+0000 7f0856f9c740 1 -- 172.21.15.129:0/256677352 wait complete. 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0856f9c740 1 -- start start 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0856f9c740 1 Processor -- start 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f9ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0856f9c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0856f9c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1a8a10 con 0x55e5c6f91a70 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0856f9c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1a9e80 con 0x55e5c6f910b0 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0856f9c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1a90e0 con 0x55e5c6f8fee0 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0856f9c740 10 client.0 _ll_register_callbacks cb 0x7f0848031920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0852749640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0851747640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0851f48640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f9ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0852749640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52134/0 (socket says 172.21.15.129:52134) 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.666+0000 7f0852749640 1 -- 172.21.15.129:0/1603992727 learned_addr learned my addr 172.21.15.129:0/1603992727 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f0852749640 1 -- 172.21.15.129:0/1603992727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 msgr2=0x55e5c6f9ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f0852749640 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f9ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f0852749640 1 -- 172.21.15.129:0/1603992727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 msgr2=0x55e5c6f9d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f0852749640 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f0852749640 1 -- 172.21.15.129:0/1603992727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c6eb54b0 con 0x55e5c6f91a70 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f0852749640 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa0430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f084409f040 tx=0x7f084409ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2310c141b5e7dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f08357fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f08357fa640 1 -- 172.21.15.129:0/1603992727 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08440b9030 con 0x55e5c6f91a70 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f08357fa640 1 -- 172.21.15.129:0/1603992727 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f084409db90 con 0x55e5c6f91a70 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f0856f9c740 1 -- 172.21.15.129:0/1603992727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083c00f8f0 con 0x55e5c6f91a70 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f08357fa640 1 -- 172.21.15.129:0/1603992727 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f084401f040 con 0x55e5c6f91a70 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f0856f9c740 10 client.16152 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f0856f9c740 1 -- 172.21.15.129:0/1603992727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e5c6fa1bd0 con 0x55e5c6f91a70 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.667+0000 7f08357fa640 1 -- 172.21.15.129:0/1603992727 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083c00f8f0 con 0x55e5c6f91a70 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f08357fa640 1 -- 172.21.15.129:0/1603992727 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f084400f040 con 0x55e5c6f91a70 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f0856f9c740 10 client.16152 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f0856f9c740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f0856f9c740 20 client.16152 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:15.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5c700c770 0x55e5c700eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:15.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f0856f9c740 4 client.16152 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:26:15.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f0856f9c740 1 -- 172.21.15.129:0/1603992727 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5c6efc800 con 0x55e5c700c770 2024-04-26T21:26:15.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f0851f48640 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5c700c770 0x55e5c700eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:15.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f08357fa640 10 client.16152 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:15.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.668+0000 7f0851f48640 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5c700c770 0x55e5c700eb60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe080ef30dfec399 server_cookie=7b46c8a8db28917 in_seq=0 out_seq=0 2024-04-26T21:26:15.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f08357fa640 1 -- 172.21.15.129:0/1603992727 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55e5c6efc800 con 0x55e5c700c770 2024-04-26T21:26:15.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f08357fa640 10 client.16152 handle_command_reply: tid=0 2024-04-26T21:26:15.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 client.16152 shutdown 2024-04-26T21:26:15.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 -- 172.21.15.129:0/1603992727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 msgr2=0x55e5c6fa0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa0430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f084409f040 tx=0x7f084409ae20 comp rx=0 tx=0).stop 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 -- 172.21.15.129:0/1603992727 shutdown_connections 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x55e5c700c770 0x55e5c700eb60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5c6f910b0 0x55e5c6f9d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5c6f91a70 0x55e5c6fa0430 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 --2- 172.21.15.129:0/1603992727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e5c6f8fee0 0x55e5c6f9ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f08357fa640 0 client.16152 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 -- 172.21.15.129:0/1603992727 >> 172.21.15.129:0/1603992727 conn(0x55e5c6f8b3d0 msgr2=0x55e5c6f8a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 -- 172.21.15.129:0/1603992727 shutdown_connections 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 1 -- 172.21.15.129:0/1603992727 wait complete. 2024-04-26T21:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.682+0000 7f0856f9c740 20 client.16152 trim_cache size 0 max 16384 2024-04-26T21:26:15.686 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:26:15.686 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "scrub active (5420 inodes in the stack)", 2024-04-26T21:26:15.686 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": { 2024-04-26T21:26:15.686 INFO:teuthology.orchestra.run.smithi129.stdout: "cbe4dae2-bfb3-4718-ab3a-199872b33243": { 2024-04-26T21:26:15.686 INFO:teuthology.orchestra.run.smithi129.stdout: "path": "/", 2024-04-26T21:26:15.686 INFO:teuthology.orchestra.run.smithi129.stdout: "tag": "cbe4dae2-bfb3-4718-ab3a-199872b33243", 2024-04-26T21:26:15.686 INFO:teuthology.orchestra.run.smithi129.stdout: "options": "recursive,force" 2024-04-26T21:26:15.686 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:15.686 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:15.687 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:26:15.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 -- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f08380a3c40 msgr2=0x7f08380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f08380a3c40 0x7f08380a6100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0848009230 tx=0x7f08480095e0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 -- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c108650 msgr2=0x7f084c1a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c108650 0x7f084c1a0190 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f084400eaa0 tx=0x7f084400ef60 comp rx=0 tx=0).stop 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 -- 172.21.15.129:0/3116179615 shutdown_connections 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f08380a3c40 0x7f08380a6100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f084c108650 0x7f084c1a0190 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f084c105d20 0x7f084c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 --2- 172.21.15.129:0/3116179615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f084c1033f0 0x7f084c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 -- 172.21.15.129:0/3116179615 >> 172.21.15.129:0/3116179615 conn(0x7f084c0fce20 msgr2=0x7f084c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 -- 172.21.15.129:0/3116179615 shutdown_connections 2024-04-26T21:26:15.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:15.683+0000 7f08541ac640 1 -- 172.21.15.129:0/3116179615 wait complete. 2024-04-26T21:26:15.695 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:15 smithi129 ceph-mon[29237]: pgmap v598: 97 pgs: 97 active+clean; 340 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 675 op/s 2024-04-26T21:26:15.696 INFO:tasks.cephfs.filesystem:scrub status for tag:cbe4dae2-bfb3-4718-ab3a-199872b33243 - {'path': '/', 'tag': 'cbe4dae2-bfb3-4718-ab3a-199872b33243', 'options': 'recursive,force'} 2024-04-26T21:26:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:15 smithi150 ceph-mon[32290]: pgmap v598: 97 pgs: 97 active+clean; 340 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 675 op/s 2024-04-26T21:26:15.892 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:15 smithi171 ceph-mon[33478]: pgmap v598: 97 pgs: 97 active+clean; 340 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 675 op/s 2024-04-26T21:26:16.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:16 smithi150 ceph-mon[32290]: scrub queued for path: / 2024-04-26T21:26:16.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:16 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:26:16.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:16 smithi150 ceph-mon[32290]: scrub summary: active paths [/] 2024-04-26T21:26:16.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:16 smithi129 ceph-mon[29237]: scrub queued for path: / 2024-04-26T21:26:16.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:16 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:26:16.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:16 smithi129 ceph-mon[29237]: scrub summary: active paths [/] 2024-04-26T21:26:16.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:16 smithi171 ceph-mon[33478]: scrub queued for path: / 2024-04-26T21:26:16.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:16 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:26:16.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:16 smithi171 ceph-mon[33478]: scrub summary: active paths [/] 2024-04-26T21:26:17.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:17 smithi150 ceph-mon[32290]: pgmap v599: 97 pgs: 97 active+clean; 328 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 646 op/s 2024-04-26T21:26:17.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:17 smithi129 ceph-mon[29237]: pgmap v599: 97 pgs: 97 active+clean; 328 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 646 op/s 2024-04-26T21:26:17.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:17 smithi171 ceph-mon[33478]: pgmap v599: 97 pgs: 97 active+clean; 328 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 646 op/s 2024-04-26T21:26:19.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:19 smithi150 ceph-mon[32290]: pgmap v600: 97 pgs: 97 active+clean; 317 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 670 op/s 2024-04-26T21:26:19.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:19 smithi129 ceph-mon[29237]: pgmap v600: 97 pgs: 97 active+clean; 317 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 670 op/s 2024-04-26T21:26:19.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:19 smithi171 ceph-mon[33478]: pgmap v600: 97 pgs: 97 active+clean; 317 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 670 op/s 2024-04-26T21:26:21.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:21 smithi150 ceph-mon[32290]: pgmap v601: 97 pgs: 97 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 704 op/s 2024-04-26T21:26:21.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:21 smithi129 ceph-mon[29237]: pgmap v601: 97 pgs: 97 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 704 op/s 2024-04-26T21:26:21.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:21 smithi171 ceph-mon[33478]: pgmap v601: 97 pgs: 97 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 704 op/s 2024-04-26T21:26:22.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:22 smithi150 ceph-mon[32290]: scrub summary: idle+waiting paths [/] 2024-04-26T21:26:22.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:22 smithi129 ceph-mon[29237]: scrub summary: idle+waiting paths [/] 2024-04-26T21:26:22.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:22 smithi171 ceph-mon[33478]: scrub summary: idle+waiting paths [/] 2024-04-26T21:26:23.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:23 smithi150 ceph-mon[32290]: pgmap v602: 97 pgs: 97 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 892 op/s 2024-04-26T21:26:23.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:23 smithi129 ceph-mon[29237]: pgmap v602: 97 pgs: 97 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 892 op/s 2024-04-26T21:26:23.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:23 smithi171 ceph-mon[33478]: pgmap v602: 97 pgs: 97 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 892 op/s 2024-04-26T21:26:25.469 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:25 smithi129 ceph-mon[29237]: scrub summary: idle 2024-04-26T21:26:25.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:25 smithi171 ceph-mon[33478]: scrub summary: idle 2024-04-26T21:26:25.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:25 smithi150 ceph-mon[32290]: scrub summary: idle 2024-04-26T21:26:26.469 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:26 smithi129 ceph-mon[29237]: pgmap v603: 97 pgs: 97 active+clean; 284 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 917 op/s 2024-04-26T21:26:26.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:26 smithi171 ceph-mon[33478]: pgmap v603: 97 pgs: 97 active+clean; 284 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 917 op/s 2024-04-26T21:26:26.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:26 smithi150 ceph-mon[32290]: pgmap v603: 97 pgs: 97 active+clean; 284 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 917 op/s 2024-04-26T21:26:27.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:27 smithi150 ceph-mon[32290]: pgmap v604: 97 pgs: 97 active+clean; 267 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 948 op/s 2024-04-26T21:26:27.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:27 smithi129 ceph-mon[29237]: pgmap v604: 97 pgs: 97 active+clean; 267 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 948 op/s 2024-04-26T21:26:27.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:27 smithi171 ceph-mon[33478]: pgmap v604: 97 pgs: 97 active+clean; 267 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 948 op/s 2024-04-26T21:26:29.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:29 smithi150 ceph-mon[32290]: pgmap v605: 97 pgs: 97 active+clean; 245 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 918 op/s 2024-04-26T21:26:29.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:29 smithi129 ceph-mon[29237]: pgmap v605: 97 pgs: 97 active+clean; 245 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 918 op/s 2024-04-26T21:26:29.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:29 smithi171 ceph-mon[33478]: pgmap v605: 97 pgs: 97 active+clean; 245 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 918 op/s 2024-04-26T21:26:31.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:31 smithi150 ceph-mon[32290]: pgmap v606: 97 pgs: 97 active+clean; 235 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 774 op/s 2024-04-26T21:26:31.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:31 smithi129 ceph-mon[29237]: pgmap v606: 97 pgs: 97 active+clean; 235 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 774 op/s 2024-04-26T21:26:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:31 smithi171 ceph-mon[33478]: pgmap v606: 97 pgs: 97 active+clean; 235 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 774 op/s 2024-04-26T21:26:33.538 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-04-26T21:26:33.539 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T21:26:33.849 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:33 smithi129 ceph-mon[29237]: pgmap v607: 97 pgs: 97 active+clean; 223 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 690 op/s 2024-04-26T21:26:33.850 DEBUG:teuthology.parallel:result is None 2024-04-26T21:26:33.851 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T21:26:33.877 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T21:26:33.877 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-26T21:26:33.888 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-26T21:26:33.889 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:33 smithi150 ceph-mon[32290]: pgmap v607: 97 pgs: 97 active+clean; 223 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 690 op/s 2024-04-26T21:26:33.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:33 smithi171 ceph-mon[33478]: pgmap v607: 97 pgs: 97 active+clean; 223 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 690 op/s 2024-04-26T21:26:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:35 smithi150 ceph-mon[32290]: pgmap v608: 97 pgs: 97 active+clean; 214 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 494 op/s 2024-04-26T21:26:35.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:35 smithi129 ceph-mon[29237]: pgmap v608: 97 pgs: 97 active+clean; 214 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 494 op/s 2024-04-26T21:26:35.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:35 smithi171 ceph-mon[33478]: pgmap v608: 97 pgs: 97 active+clean; 214 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 494 op/s 2024-04-26T21:26:37.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:36 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:37.219 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:36 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:37.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:36 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:38.121 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:38.121 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:38.121 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:37 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:26:38.121 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:37 smithi129 ceph-mon[29237]: pgmap v609: 97 pgs: 97 active+clean; 210 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 414 op/s 2024-04-26T21:26:38.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:38.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:38.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:37 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:26:38.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:37 smithi150 ceph-mon[32290]: pgmap v609: 97 pgs: 97 active+clean; 210 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 414 op/s 2024-04-26T21:26:38.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:38.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:38.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:37 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' 2024-04-26T21:26:38.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:37 smithi171 ceph-mon[33478]: pgmap v609: 97 pgs: 97 active+clean; 210 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 414 op/s 2024-04-26T21:26:39.885 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:39 smithi150 ceph-mon[32290]: pgmap v610: 97 pgs: 97 active+clean; 208 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 336 op/s 2024-04-26T21:26:39.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:39 smithi129 ceph-mon[29237]: pgmap v610: 97 pgs: 97 active+clean; 208 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 336 op/s 2024-04-26T21:26:39.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:39 smithi171 ceph-mon[33478]: pgmap v610: 97 pgs: 97 active+clean; 208 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 336 op/s 2024-04-26T21:26:41.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:41 smithi150 ceph-mon[32290]: pgmap v611: 97 pgs: 97 active+clean; 208 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 792 KiB/s wr, 251 op/s 2024-04-26T21:26:41.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:41 smithi129 ceph-mon[29237]: pgmap v611: 97 pgs: 97 active+clean; 208 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 792 KiB/s wr, 251 op/s 2024-04-26T21:26:41.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:41 smithi171 ceph-mon[33478]: pgmap v611: 97 pgs: 97 active+clean; 208 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 792 KiB/s wr, 251 op/s 2024-04-26T21:26:43.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:43 smithi150 ceph-mon[32290]: pgmap v612: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 782 KiB/s wr, 200 op/s 2024-04-26T21:26:43.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:43 smithi129 ceph-mon[29237]: pgmap v612: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 782 KiB/s wr, 200 op/s 2024-04-26T21:26:43.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:43 smithi171 ceph-mon[33478]: pgmap v612: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 782 KiB/s wr, 200 op/s 2024-04-26T21:26:45.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:26:45.697 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:26:45.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.807+0000 7fddde4f1640 1 -- 172.21.15.129:0/1857958030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8103450 msgr2=0x7fddd8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.807+0000 7fddde4f1640 1 --2- 172.21.15.129:0/1857958030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8103450 0x7fddd8105840 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009900 tx=0x7fddc802a410 comp rx=0 tx=0).stop 2024-04-26T21:26:45.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.807+0000 7fddde4f1640 1 -- 172.21.15.129:0/1857958030 shutdown_connections 2024-04-26T21:26:45.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.807+0000 7fddde4f1640 1 --2- 172.21.15.129:0/1857958030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd81086b0 0x7fddd810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.807+0000 7fddde4f1640 1 --2- 172.21.15.129:0/1857958030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddd8105d80 0x7fddd8108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.807+0000 7fddde4f1640 1 --2- 172.21.15.129:0/1857958030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8103450 0x7fddd8105840 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.807+0000 7fddde4f1640 1 -- 172.21.15.129:0/1857958030 >> 172.21.15.129:0/1857958030 conn(0x7fddd80fcde0 msgr2=0x7fddd80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 -- 172.21.15.129:0/1857958030 shutdown_connections 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 -- 172.21.15.129:0/1857958030 wait complete. 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 Processor -- start 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 -- start start 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddd8103450 0x7fddd8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd8105d80 0x7fddd8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81086b0 0x7fddd81a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fddd810e4c0 con 0x7fddd81086b0 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fddd810e340 con 0x7fddd8105d80 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.808+0000 7fddde4f1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fddd810e640 con 0x7fddd8103450 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fdddca8e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81086b0 0x7fddd81a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fdddca8e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81086b0 0x7fddd81a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44590/0 (socket says 172.21.15.129:44590) 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fdddca8e640 1 -- 172.21.15.129:0/3326279048 learned_addr learned my addr 172.21.15.129:0/3326279048 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fddd77fe640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd8105d80 0x7fddd8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fddd7fff640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddd8103450 0x7fddd8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fdddca8e640 1 -- 172.21.15.129:0/3326279048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddd8103450 msgr2=0x7fddd8198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fdddca8e640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddd8103450 0x7fddd8198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fdddca8e640 1 -- 172.21.15.129:0/3326279048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd8105d80 msgr2=0x7fddd8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fdddca8e640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd8105d80 0x7fddd8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fdddca8e640 1 -- 172.21.15.129:0/3326279048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd81a0850 con 0x7fddd81086b0 2024-04-26T21:26:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fdddca8e640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81086b0 0x7fddd81a01b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fddd000eaa0 tx=0x7fddd000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fddd57fa640 1 -- 172.21.15.129:0/3326279048 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd000cdb0 con 0x7fddd81086b0 2024-04-26T21:26:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fddd57fa640 1 -- 172.21.15.129:0/3326279048 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd0005020 con 0x7fddd81086b0 2024-04-26T21:26:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fddde4f1640 1 -- 172.21.15.129:0/3326279048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd81a0b40 con 0x7fddd81086b0 2024-04-26T21:26:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fddd57fa640 1 -- 172.21.15.129:0/3326279048 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd00116d0 con 0x7fddd81086b0 2024-04-26T21:26:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.809+0000 7fddde4f1640 1 -- 172.21.15.129:0/3326279048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd81a1040 con 0x7fddd81086b0 2024-04-26T21:26:45.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd57fa640 1 -- 172.21.15.129:0/3326279048 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fddd00051d0 con 0x7fddd81086b0 2024-04-26T21:26:45.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fdde1207740 1 Processor -- start 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fdde1207740 1 -- start start 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fdde1207740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b0426300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fdde1207740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b04270b0 0x5616b0427530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fdde1207740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0427a70 0x5616b042e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fdde1207740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616b038ad80 con 0x5616b0425ee0 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fdde1207740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5616b03448b0 con 0x5616b04270b0 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fdde1207740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5616b0386d40 con 0x5616b0427a70 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b0426300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b0426300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44604/0 (socket says 172.21.15.129:44604) 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd77fe640 1 -- 172.21.15.129:0/2159264766 learned_addr learned my addr 172.21.15.129:0/2159264766 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd57fa640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fddc00a3c40 0x7fddc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd7fff640 1 --2- 172.21.15.129:0/2159264766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b04270b0 0x5616b0427530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fdddca8e640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fddc00a3c40 0x7fddc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd77fe640 1 --2- 172.21.15.129:0/2159264766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0427a70 0x5616b042e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd77fe640 1 -- 172.21.15.129:0/2159264766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0427a70 msgr2=0x5616b042e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd77fe640 1 --2- 172.21.15.129:0/2159264766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0427a70 0x5616b042e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd77fe640 1 -- 172.21.15.129:0/2159264766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b04270b0 msgr2=0x5616b0427530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.812+0000 7fddd77fe640 1 --2- 172.21.15.129:0/2159264766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b04270b0 0x5616b0427530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.813+0000 7fddd77fe640 1 -- 172.21.15.129:0/2159264766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b0315a40 con 0x5616b0425ee0 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.813+0000 7fddd57fa640 1 -- 172.21.15.129:0/3326279048 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fddd00d0790 con 0x7fddd81086b0 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.813+0000 7fddd77fe640 1 --2- 172.21.15.129:0/2159264766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b0426300 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00b8f0 tx=0x7fddcc004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bef0c432679751 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.813+0000 7fdddca8e640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fddc00a3c40 0x7fddc00a6100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fddd00a5040 tx=0x7fddd006f460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.813+0000 7fddbe7fc640 1 -- 172.21.15.129:0/2159264766 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc003d70 con 0x5616b0425ee0 2024-04-26T21:26:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.813+0000 7fddbe7fc640 1 -- 172.21.15.129:0/2159264766 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddcc0054d0 con 0x5616b0425ee0 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.813+0000 7fddbe7fc640 1 -- 172.21.15.129:0/2159264766 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc010460 con 0x5616b0425ee0 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 -- 172.21.15.129:0/2159264766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 msgr2=0x5616b0426300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 --2- 172.21.15.129:0/2159264766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b0426300 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00b8f0 tx=0x7fddcc004250 comp rx=0 tx=0).stop 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 -- 172.21.15.129:0/2159264766 shutdown_connections 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 --2- 172.21.15.129:0/2159264766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b04270b0 0x5616b0427530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 --2- 172.21.15.129:0/2159264766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0427a70 0x5616b042e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 --2- 172.21.15.129:0/2159264766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b0426300 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 -- 172.21.15.129:0/2159264766 >> 172.21.15.129:0/2159264766 conn(0x5616b04213d0 msgr2=0x5616b04217d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 -- 172.21.15.129:0/2159264766 shutdown_connections 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 -- 172.21.15.129:0/2159264766 wait complete. 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 -- start start 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 Processor -- start 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0425ee0 0x5616b0430390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b04308d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616b042ea80 con 0x5616b04270b0 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5616b0311d10 con 0x5616b0427a70 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5616b042ec00 con 0x5616b0425ee0 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.814+0000 7fdde1207740 10 client.0 _ll_register_callbacks cb 0x5616b033e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b04308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b04308d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44606/0 (socket says 172.21.15.129:44606) 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd7fff640 1 -- 172.21.15.129:0/4141584936 learned_addr learned my addr 172.21.15.129:0/4141584936 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd77fe640 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd77fe640 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0425ee0 0x5616b0430390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd7fff640 1 -- 172.21.15.129:0/4141584936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0425ee0 msgr2=0x5616b0430390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd7fff640 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0425ee0 0x5616b0430390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd7fff640 1 -- 172.21.15.129:0/4141584936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 msgr2=0x5616b04378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd7fff640 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd7fff640 1 -- 172.21.15.129:0/4141584936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b01667d0 con 0x5616b04270b0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd77fe640 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddd7fff640 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b04308d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005c40 tx=0x7fddc8031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c46c69ba44fe5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddb77fe640 1 -- 172.21.15.129:0/4141584936 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8035040 con 0x5616b04270b0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddb77fe640 1 -- 172.21.15.129:0/4141584936 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fddc8004040 con 0x5616b04270b0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fdde1207740 1 -- 172.21.15.129:0/4141584936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddcc0054d0 con 0x5616b04270b0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fddb77fe640 1 -- 172.21.15.129:0/4141584936 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fddc8004240 con 0x5616b04270b0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fdde1207740 10 client.16164 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.815+0000 7fdde1207740 1 -- 172.21.15.129:0/4141584936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5616b009c3f0 con 0x5616b04270b0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fddb77fe640 1 -- 172.21.15.129:0/4141584936 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddcc0054d0 con 0x5616b04270b0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fddb77fe640 1 -- 172.21.15.129:0/4141584936 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fddc8044020 con 0x5616b04270b0 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fdde1207740 10 client.16164 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fdde1207740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fdde1207740 20 client.16164 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fdde1207740 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5616b043bde0 0x5616b043e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fdde1207740 4 client.16164 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fdde1207740 1 -- 172.21.15.129:0/4141584936 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b0398990 con 0x5616b043bde0 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fddd77fe640 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5616b043bde0 0x5616b043e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fddb77fe640 10 client.16164 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.816+0000 7fddd77fe640 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5616b043bde0 0x5616b043e1f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e909e5df70fb6b server_cookie=7e0cc59278ccdd63 in_seq=0 out_seq=0 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fddb77fe640 1 -- 172.21.15.129:0/4141584936 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616b0398990 con 0x5616b043bde0 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fddb77fe640 10 client.16164 handle_command_reply: tid=0 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fdde1207740 1 client.16164 shutdown 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fdde1207740 1 -- 172.21.15.129:0/4141584936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 msgr2=0x5616b04308d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fdde1207740 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b04308d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005c40 tx=0x7fddc8031f10 comp rx=0 tx=0).stop 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fdde1207740 1 -- 172.21.15.129:0/4141584936 shutdown_connections 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fdde1207740 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5616b043bde0 0x5616b043e1f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fdde1207740 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fdde1207740 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b04308d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fdde1207740 1 --2- 172.21.15.129:0/4141584936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0425ee0 0x5616b0430390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fddb77fe640 0 client.16164 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.817+0000 7fdde1207740 1 -- 172.21.15.129:0/4141584936 >> 172.21.15.129:0/4141584936 conn(0x5616b04213d0 msgr2=0x5616b0420550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.818+0000 7fdde1207740 1 -- 172.21.15.129:0/4141584936 shutdown_connections 2024-04-26T21:26:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.818+0000 7fdde1207740 1 -- 172.21.15.129:0/4141584936 wait complete. 2024-04-26T21:26:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.818+0000 7fdde1207740 20 client.16164 trim_cache size 0 max 16384 2024-04-26T21:26:45.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fdde1207740 1 Processor -- start 2024-04-26T21:26:45.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fdde1207740 1 -- start start 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fdde1207740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0425ee0 0x5616b042ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fdde1207740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b042b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fdde1207740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fdde1207740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fddd81a7d70 con 0x5616b04270b0 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fdde1207740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fddd81aa5f0 con 0x5616b0427a70 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fdde1207740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fddd81a83e0 con 0x5616b0425ee0 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fddd7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b042b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fddd7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b042b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44618/0 (socket says 172.21.15.129:44618) 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fddd7fff640 1 -- 172.21.15.129:0/3645482561 learned_addr learned my addr 172.21.15.129:0/3645482561 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fddd7fff640 1 -- 172.21.15.129:0/3645482561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0425ee0 msgr2=0x5616b042ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fddd7fff640 1 --2- 172.21.15.129:0/3645482561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0425ee0 0x5616b042ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fddd7fff640 1 -- 172.21.15.129:0/3645482561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 msgr2=0x5616b04240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fddd7fff640 1 --2- 172.21.15.129:0/3645482561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fddd7fff640 1 -- 172.21.15.129:0/3645482561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b005c0f0 con 0x5616b04270b0 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.835+0000 7fddd7fff640 1 --2- 172.21.15.129:0/3645482561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b042b0d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009850 tx=0x7fddc8031a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c38b967cf6bf17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fddbdffb640 1 -- 172.21.15.129:0/3645482561 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8035040 con 0x5616b04270b0 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fddbdffb640 1 -- 172.21.15.129:0/3645482561 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc8046780 con 0x5616b04270b0 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fddbdffb640 1 -- 172.21.15.129:0/3645482561 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc803cca0 con 0x5616b04270b0 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 -- 172.21.15.129:0/3645482561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 msgr2=0x5616b042b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 --2- 172.21.15.129:0/3645482561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b042b0d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009850 tx=0x7fddc8031a40 comp rx=0 tx=0).stop 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 -- 172.21.15.129:0/3645482561 shutdown_connections 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 --2- 172.21.15.129:0/3645482561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 --2- 172.21.15.129:0/3645482561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b04270b0 0x5616b042b0d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 --2- 172.21.15.129:0/3645482561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b0425ee0 0x5616b042ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 -- 172.21.15.129:0/3645482561 >> 172.21.15.129:0/3645482561 conn(0x5616b04213d0 msgr2=0x5616b041c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:45.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 -- 172.21.15.129:0/3645482561 shutdown_connections 2024-04-26T21:26:45.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 -- 172.21.15.129:0/3645482561 wait complete. 2024-04-26T21:26:45.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.836+0000 7fdde1207740 1 -- start start 2024-04-26T21:26:45.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fdde1207740 1 Processor -- start 2024-04-26T21:26:45.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fdde1207740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b049d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fdde1207740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b04270b0 0x5616b049d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fdde1207740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04a4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fdde1207740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fddd81a8ab0 con 0x5616b0425ee0 2024-04-26T21:26:45.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fdde1207740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fddd81a9f20 con 0x5616b0427a70 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fdde1207740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fddd81a9180 con 0x5616b04270b0 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fdde1207740 10 client.0 _ll_register_callbacks cb 0x7fddc8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b049d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b049d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44626/0 (socket says 172.21.15.129:44626) 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd77fe640 1 -- 172.21.15.129:0/2229858081 learned_addr learned my addr 172.21.15.129:0/2229858081 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd77fe640 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd7fff640 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b04270b0 0x5616b049d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd77fe640 1 -- 172.21.15.129:0/2229858081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b04270b0 msgr2=0x5616b049d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd77fe640 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b04270b0 0x5616b049d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd77fe640 1 -- 172.21.15.129:0/2229858081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 msgr2=0x5616b04a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd77fe640 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.837+0000 7fddd77fe640 1 -- 172.21.15.129:0/2229858081 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b034b4b0 con 0x5616b0425ee0 2024-04-26T21:26:45.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fddd77fe640 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b049d3b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00b490 tx=0x7fddcc00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fad3a852d0651c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fddb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fddb77fe640 1 -- 172.21.15.129:0/2229858081 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc02b2d0 con 0x5616b0425ee0 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fddb77fe640 1 -- 172.21.15.129:0/2229858081 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fddcc02b470 con 0x5616b0425ee0 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fdde1207740 1 -- 172.21.15.129:0/2229858081 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddc8046780 con 0x5616b0425ee0 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fddb77fe640 1 -- 172.21.15.129:0/2229858081 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fddcc02b670 con 0x5616b0425ee0 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fdde1207740 10 client.16170 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fdde1207740 1 -- 172.21.15.129:0/2229858081 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5616b04a60d0 con 0x5616b0425ee0 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fddb77fe640 1 -- 172.21.15.129:0/2229858081 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc8046780 con 0x5616b0425ee0 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fddb77fe640 1 -- 172.21.15.129:0/2229858081 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fddcc01c420 con 0x5616b0425ee0 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fdde1207740 10 client.16170 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fdde1207740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fdde1207740 20 client.16170 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fdde1207740 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5616b04a6960 0x5616b04a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fdde1207740 4 client.16170 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.838+0000 7fdde1207740 1 -- 172.21.15.129:0/2229858081 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5616b0392800 con 0x5616b04a6960 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fddd7fff640 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5616b04a6960 0x5616b04a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fddb77fe640 10 client.16170 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fddd7fff640 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5616b04a6960 0x5616b04a8d80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83880ffb25846204 server_cookie=aa550bf1c1b4c437 in_seq=0 out_seq=0 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fddb77fe640 1 -- 172.21.15.129:0/2229858081 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5616b0392800 con 0x5616b04a6960 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fddb77fe640 10 client.16170 handle_command_reply: tid=0 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 client.16170 shutdown 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 -- 172.21.15.129:0/2229858081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 msgr2=0x5616b049d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b049d3b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00b490 tx=0x7fddcc00b420 comp rx=0 tx=0).stop 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 -- 172.21.15.129:0/2229858081 shutdown_connections 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5616b04a6960 0x5616b04a8d80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616b0427a70 0x5616b04a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616b04270b0 0x5616b049d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fddb77fe640 0 client.16170 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 --2- 172.21.15.129:0/2229858081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616b0425ee0 0x5616b049d3b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 -- 172.21.15.129:0/2229858081 >> 172.21.15.129:0/2229858081 conn(0x5616b04213d0 msgr2=0x5616b04302b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 -- 172.21.15.129:0/2229858081 shutdown_connections 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 1 -- 172.21.15.129:0/2229858081 wait complete. 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.839+0000 7fdde1207740 20 client.16170 trim_cache size 0 max 16384 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stdout: "status": "no active scrubs running", 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stdout: "scrubs": {} 2024-04-26T21:26:45.848 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:26:45.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 -- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fddc00a3c40 msgr2=0x7fddc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fddc00a3c40 0x7fddc00a6100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fddd00a5040 tx=0x7fddd006f460 comp rx=0 tx=0).stop 2024-04-26T21:26:45.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 -- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81086b0 msgr2=0x7fddd81a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81086b0 0x7fddd81a01b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fddd000eaa0 tx=0x7fddd000ef60 comp rx=0 tx=0).stop 2024-04-26T21:26:45.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 -- 172.21.15.129:0/3326279048 shutdown_connections 2024-04-26T21:26:45.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fddc00a3c40 0x7fddc00a6100 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd8105d80 0x7fddd8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81086b0 0x7fddd81a01b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 --2- 172.21.15.129:0/3326279048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddd8103450 0x7fddd8198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 -- 172.21.15.129:0/3326279048 >> 172.21.15.129:0/3326279048 conn(0x7fddd80fcde0 msgr2=0x7fddd81052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:45.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 -- 172.21.15.129:0/3326279048 shutdown_connections 2024-04-26T21:26:45.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.840+0000 7fddde4f1640 1 -- 172.21.15.129:0/3326279048 wait complete. 2024-04-26T21:26:45.851 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:45 smithi129 ceph-mon[29237]: pgmap v613: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 422 KiB/s wr, 113 op/s 2024-04-26T21:26:45.852 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:26:45.852 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:26:45.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:45 smithi150 ceph-mon[32290]: pgmap v613: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 422 KiB/s wr, 113 op/s 2024-04-26T21:26:45.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 -- 172.21.15.129:0/121079406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc101880 msgr2=0x7f60fc101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 --2- 172.21.15.129:0/121079406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc101880 0x7f60fc101ca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009900 tx=0x7f60f002a410 comp rx=0 tx=0).stop 2024-04-26T21:26:45.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 -- 172.21.15.129:0/121079406 shutdown_connections 2024-04-26T21:26:45.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 --2- 172.21.15.129:0/121079406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60fc102a70 0x7f60fc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 --2- 172.21.15.129:0/121079406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60fc103430 0x7f60fc109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 --2- 172.21.15.129:0/121079406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc101880 0x7f60fc101ca0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 -- 172.21.15.129:0/121079406 >> 172.21.15.129:0/121079406 conn(0x7f60fc0fce10 msgr2=0x7f60fc0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:45.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 -- 172.21.15.129:0/121079406 shutdown_connections 2024-04-26T21:26:45.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 -- 172.21.15.129:0/121079406 wait complete. 2024-04-26T21:26:45.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 Processor -- start 2024-04-26T21:26:45.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.950+0000 7f6101511640 1 -- start start 2024-04-26T21:26:45.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f6101511640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60fc101880 0x7f60fc10c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f6101511640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc102a70 0x7f60fc10a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f6101511640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60fc103430 0x7f60fc10ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f6101511640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc10e4d0 con 0x7f60fc102a70 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f6101511640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc10e350 con 0x7f60fc103430 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f6101511640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc10e650 con 0x7f60fc101880 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fa7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc102a70 0x7f60fc10a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fa7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc102a70 0x7f60fc10a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44650/0 (socket says 172.21.15.129:44650) 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fa7fc640 1 -- 172.21.15.129:0/3921751649 learned_addr learned my addr 172.21.15.129:0/3921751649 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fb7fe640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60fc103430 0x7f60fc10ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fa7fc640 1 -- 172.21.15.129:0/3921751649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60fc101880 msgr2=0x7f60fc10c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fa7fc640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60fc101880 0x7f60fc10c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fa7fc640 1 -- 172.21.15.129:0/3921751649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60fc103430 msgr2=0x7f60fc10ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:45.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fa7fc640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60fc103430 0x7f60fc10ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:45.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fa7fc640 1 -- 172.21.15.129:0/3921751649 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc10b610 con 0x7f60fc102a70 2024-04-26T21:26:45.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60fa7fc640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc102a70 0x7f60fc10a840 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c920 tx=0x7f60ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:45.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60e3fff640 1 -- 172.21.15.129:0/3921751649 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec007c70 con 0x7f60fc102a70 2024-04-26T21:26:45.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60e3fff640 1 -- 172.21.15.129:0/3921751649 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ec007e10 con 0x7f60fc102a70 2024-04-26T21:26:45.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc1a2c50 con 0x7f60fc102a70 2024-04-26T21:26:45.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f60e3fff640 1 -- 172.21.15.129:0/3921751649 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec018820 con 0x7f60fc102a70 2024-04-26T21:26:45.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.951+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1a31a0 con 0x7f60fc102a70 2024-04-26T21:26:45.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.952+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc108c20 con 0x7f60fc102a70 2024-04-26T21:26:45.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.953+0000 7f60e3fff640 1 -- 172.21.15.129:0/3921751649 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f60ec00f450 con 0x7f60fc102a70 2024-04-26T21:26:45.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.954+0000 7f60e3fff640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60e40a3c40 0x7f60e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:45.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.954+0000 7f60faffd640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60e40a3c40 0x7f60e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:45.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.954+0000 7f60e3fff640 1 -- 172.21.15.129:0/3921751649 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f60ec0d07d0 con 0x7f60fc102a70 2024-04-26T21:26:45.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.954+0000 7f60faffd640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60e40a3c40 0x7f60e40a6100 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f60f00096c0 tx=0x7f60f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:45.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:45.958+0000 7f60e3fff640 1 -- 172.21.15.129:0/3921751649 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f60ec014030 con 0x7f60fc102a70 2024-04-26T21:26:45.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:45 smithi171 ceph-mon[33478]: pgmap v613: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 422 KiB/s wr, 113 op/s 2024-04-26T21:26:46.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.174+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60fc02dad0 con 0x7f60fc102a70 2024-04-26T21:26:46.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.174+0000 7f60e3fff640 1 -- 172.21.15.129:0/3921751649 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f60ec00de20 con 0x7f60fc102a70 2024-04-26T21:26:46.176 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:26:46.176 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:26:46.176 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60e40a3c40 msgr2=0x7f60e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60e40a3c40 0x7f60e40a6100 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f60f00096c0 tx=0x7f60f0000bc0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc102a70 msgr2=0x7f60fc10a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc102a70 0x7f60fc10a840 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c920 tx=0x7f60ec00cde0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 shutdown_connections 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f60e40a3c40 0x7f60e40a6100 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60fc103430 0x7f60fc10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc102a70 0x7f60fc10a840 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 --2- 172.21.15.129:0/3921751649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60fc101880 0x7f60fc10c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.176+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 >> 172.21.15.129:0/3921751649 conn(0x7f60fc0fce10 msgr2=0x7f60fc105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:46.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.177+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 shutdown_connections 2024-04-26T21:26:46.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.177+0000 7f6101511640 1 -- 172.21.15.129:0/3921751649 wait complete. 2024-04-26T21:26:46.189 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:26:46.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 -- 172.21.15.129:0/4212263782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84101870 msgr2=0x7fef84101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 --2- 172.21.15.129:0/4212263782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84101870 0x7fef84101c90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fef78009900 tx=0x7fef7802a410 comp rx=0 tx=0).stop 2024-04-26T21:26:46.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 -- 172.21.15.129:0/4212263782 shutdown_connections 2024-04-26T21:26:46.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 --2- 172.21.15.129:0/4212263782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84102a60 0x7fef84102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 --2- 172.21.15.129:0/4212263782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef84103420 0x7fef84109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 --2- 172.21.15.129:0/4212263782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84101870 0x7fef84101c90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 -- 172.21.15.129:0/4212263782 >> 172.21.15.129:0/4212263782 conn(0x7fef840fcde0 msgr2=0x7fef840ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:46.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 -- 172.21.15.129:0/4212263782 shutdown_connections 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 -- 172.21.15.129:0/4212263782 wait complete. 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 Processor -- start 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.288+0000 7fef8ae25640 1 -- start start 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef8ae25640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef84101870 0x7fef84197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef8ae25640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84102a60 0x7fef84198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef8ae25640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84103420 0x7fef8419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef8ae25640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef8410d3a0 con 0x7fef84103420 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef8ae25640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef8410d220 con 0x7fef84102a60 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef8ae25640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fef8410d520 con 0x7fef84101870 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef893c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84103420 0x7fef8419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef893c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84103420 0x7fef8419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44658/0 (socket says 172.21.15.129:44658) 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef893c2640 1 -- 172.21.15.129:0/3987706763 learned_addr learned my addr 172.21.15.129:0/3987706763 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:46.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef88bc1640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef84101870 0x7fef84197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef83fff640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84102a60 0x7fef84198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef893c2640 1 -- 172.21.15.129:0/3987706763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef84101870 msgr2=0x7fef84197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef893c2640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef84101870 0x7fef84197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef893c2640 1 -- 172.21.15.129:0/3987706763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84102a60 msgr2=0x7fef84198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef893c2640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84102a60 0x7fef84198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef893c2640 1 -- 172.21.15.129:0/3987706763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8419f6f0 con 0x7fef84103420 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef893c2640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84103420 0x7fef8419f080 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00eaa0 tx=0x7fef7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef81ffb640 1 -- 172.21.15.129:0/3987706763 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00cdb0 con 0x7fef84103420 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef81ffb640 1 -- 172.21.15.129:0/3987706763 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c005020 con 0x7fef84103420 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef8ae25640 1 -- 172.21.15.129:0/3987706763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8419f9e0 con 0x7fef84103420 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.289+0000 7fef81ffb640 1 -- 172.21.15.129:0/3987706763 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0116d0 con 0x7fef84103420 2024-04-26T21:26:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.290+0000 7fef8ae25640 1 -- 172.21.15.129:0/3987706763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8419ff80 con 0x7fef84103420 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef81ffb640 1 -- 172.21.15.129:0/3987706763 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fef7c0051d0 con 0x7fef84103420 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef8dc1a740 1 Processor -- start 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef8dc1a740 1 -- start start 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 0x5604092a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092a3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a35c0 0x5604092a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef81ffb640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef740a3cc0 0x7fef740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560409203d80 con 0x5604092a2c00 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5604091bda20 con 0x5604092a1a30 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5604091ffd40 con 0x5604092a35c0 2024-04-26T21:26:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef88bc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef83fff640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef740a3cc0 0x7fef740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef88bc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092a3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44660/0 (socket says 172.21.15.129:44660) 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef88bc1640 1 -- 172.21.15.129:0/1485998470 learned_addr learned my addr 172.21.15.129:0/1485998470 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef81ffb640 1 -- 172.21.15.129:0/3987706763 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fef7c0d0650 con 0x7fef84103420 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef83fff640 1 --2- 172.21.15.129:0/1485998470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 0x5604092a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.292+0000 7fef893c2640 1 --2- 172.21.15.129:0/1485998470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a35c0 0x5604092a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef88bc1640 1 -- 172.21.15.129:0/1485998470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a35c0 msgr2=0x5604092a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef88bc1640 1 --2- 172.21.15.129:0/1485998470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a35c0 0x5604092a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef88bc1640 1 -- 172.21.15.129:0/1485998470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 msgr2=0x5604092a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef88bc1640 1 --2- 172.21.15.129:0/1485998470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 0x5604092a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef88bc1640 1 -- 172.21.15.129:0/1485998470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604091c8c30 con 0x5604092a2c00 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef83fff640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef740a3cc0 0x7fef740a6180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fef84198fe0 tx=0x7fef7000cbe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef88bc1640 1 --2- 172.21.15.129:0/1485998470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092a3080 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fef78009900 tx=0x7fef78031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fd84e3744b7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef6effd640 1 -- 172.21.15.129:0/1485998470 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78035030 con 0x5604092a2c00 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef6effd640 1 -- 172.21.15.129:0/1485998470 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef78040070 con 0x5604092a2c00 2024-04-26T21:26:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.293+0000 7fef6effd640 1 -- 172.21.15.129:0/1485998470 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78003880 con 0x5604092a2c00 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 -- 172.21.15.129:0/1485998470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 msgr2=0x5604092a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/1485998470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092a3080 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fef78009900 tx=0x7fef78031b10 comp rx=0 tx=0).stop 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 -- 172.21.15.129:0/1485998470 shutdown_connections 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/1485998470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a35c0 0x5604092a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/1485998470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092a3080 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/1485998470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 0x5604092a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 -- 172.21.15.129:0/1485998470 >> 172.21.15.129:0/1485998470 conn(0x5604091e0910 msgr2=0x56040929b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 -- 172.21.15.129:0/1485998470 shutdown_connections 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 -- 172.21.15.129:0/1485998470 wait complete. 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 -- start start 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.294+0000 7fef8dc1a740 1 Processor -- start 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a1a30 0x5604092abc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a2c00 0x5604092ac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a35c0 0x5604092b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5604092aa570 con 0x5604092a1a30 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56040918adc0 con 0x5604092a35c0 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5604092aa6f0 con 0x5604092a2c00 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 10 client.0 _ll_register_callbacks cb 0x5604091a6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef88bc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a1a30 0x5604092abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef88bc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a1a30 0x5604092abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44662/0 (socket says 172.21.15.129:44662) 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef88bc1640 1 -- 172.21.15.129:0/846146949 learned_addr learned my addr 172.21.15.129:0/846146949 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef83fff640 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a2c00 0x5604092ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef893c2640 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a35c0 0x5604092b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef88bc1640 1 -- 172.21.15.129:0/846146949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a2c00 msgr2=0x5604092ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef88bc1640 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a2c00 0x5604092ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef88bc1640 1 -- 172.21.15.129:0/846146949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a35c0 msgr2=0x5604092b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef88bc1640 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a35c0 0x5604092b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef88bc1640 1 -- 172.21.15.129:0/846146949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560408f153f0 con 0x5604092a1a30 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef88bc1640 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a1a30 0x5604092abc20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fef78000c00 tx=0x7fef78004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa299dd19c0aae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef63fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef63fff640 1 -- 172.21.15.129:0/846146949 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78035040 con 0x5604092a1a30 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef63fff640 1 -- 172.21.15.129:0/846146949 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fef780317a0 con 0x5604092a1a30 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 1 -- 172.21.15.129:0/846146949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef78040070 con 0x5604092a1a30 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef63fff640 1 -- 172.21.15.129:0/846146949 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fef78003660 con 0x5604092a1a30 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 10 client.16188 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef8dc1a740 1 -- 172.21.15.129:0/846146949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560408fdf7d0 con 0x5604092a1a30 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.295+0000 7fef63fff640 1 -- 172.21.15.129:0/846146949 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef78040070 con 0x5604092a1a30 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef63fff640 1 -- 172.21.15.129:0/846146949 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fef78045860 con 0x5604092a1a30 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef8dc1a740 10 client.16188 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef8dc1a740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef8dc1a740 20 client.16188 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5604092b7690 0x5604092b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef8dc1a740 4 client.16188 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:26:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef8dc1a740 1 -- 172.21.15.129:0/846146949 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560409211990 con 0x5604092b7690 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef83fff640 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5604092b7690 0x5604092b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef63fff640 10 client.16188 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.296+0000 7fef83fff640 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5604092b7690 0x5604092b9a80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5752e67fa9eb812c server_cookie=2352c9ab9321b59b in_seq=0 out_seq=0 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef63fff640 1 -- 172.21.15.129:0/846146949 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560409211990 con 0x5604092b7690 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef63fff640 10 client.16188 handle_command_reply: tid=0 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 client.16188 shutdown 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 -- 172.21.15.129:0/846146949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a1a30 msgr2=0x5604092abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a1a30 0x5604092abc20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fef78000c00 tx=0x7fef78004040 comp rx=0 tx=0).stop 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 -- 172.21.15.129:0/846146949 shutdown_connections 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x5604092b7690 0x5604092b9a80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a35c0 0x5604092b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef63fff640 0 client.16188 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a2c00 0x5604092ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/846146949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a1a30 0x5604092abc20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 -- 172.21.15.129:0/846146949 >> 172.21.15.129:0/846146949 conn(0x5604091e0910 msgr2=0x56040929a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 -- 172.21.15.129:0/846146949 shutdown_connections 2024-04-26T21:26:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.297+0000 7fef8dc1a740 1 -- 172.21.15.129:0/846146949 wait complete. 2024-04-26T21:26:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.298+0000 7fef8dc1a740 20 client.16188 trim_cache size 0 max 16384 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef8dc1a740 1 Processor -- start 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef8dc1a740 1 -- start start 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a1a30 0x56040929e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a2c00 0x5604092b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a35c0 0x5604092ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef841a5bf0 con 0x5604092a35c0 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef841a8470 con 0x5604092a2c00 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fef841a6260 con 0x5604092a1a30 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef893c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a35c0 0x5604092ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef893c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a35c0 0x5604092ab240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44668/0 (socket says 172.21.15.129:44668) 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef893c2640 1 -- 172.21.15.129:0/2161049747 learned_addr learned my addr 172.21.15.129:0/2161049747 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef83fff640 1 --2- 172.21.15.129:0/2161049747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a2c00 0x5604092b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef893c2640 1 -- 172.21.15.129:0/2161049747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a1a30 msgr2=0x56040929e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef893c2640 1 --2- 172.21.15.129:0/2161049747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a1a30 0x56040929e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef893c2640 1 -- 172.21.15.129:0/2161049747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a2c00 msgr2=0x5604092b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef893c2640 1 --2- 172.21.15.129:0/2161049747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a2c00 0x5604092b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef893c2640 1 -- 172.21.15.129:0/2161049747 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604092968b0 con 0x5604092a35c0 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef893c2640 1 --2- 172.21.15.129:0/2161049747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a35c0 0x5604092ab240 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fef7c09c750 tx=0x7fef7c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07c733652f1458a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef6e7fc640 1 -- 172.21.15.129:0/2161049747 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c01f040 con 0x5604092a35c0 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef6e7fc640 1 -- 172.21.15.129:0/2161049747 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c09c7c0 con 0x5604092a35c0 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.311+0000 7fef6e7fc640 1 -- 172.21.15.129:0/2161049747 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c098e30 con 0x5604092a35c0 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 -- 172.21.15.129:0/2161049747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a35c0 msgr2=0x5604092ab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/2161049747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a35c0 0x5604092ab240 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fef7c09c750 tx=0x7fef7c087040 comp rx=0 tx=0).stop 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 -- 172.21.15.129:0/2161049747 shutdown_connections 2024-04-26T21:26:46.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/2161049747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a2c00 0x5604092b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/2161049747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a35c0 0x5604092ab240 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/2161049747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a1a30 0x56040929e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 -- 172.21.15.129:0/2161049747 >> 172.21.15.129:0/2161049747 conn(0x5604091e0910 msgr2=0x56040929e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 -- 172.21.15.129:0/2161049747 shutdown_connections 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 -- 172.21.15.129:0/2161049747 wait complete. 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 -- start start 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.312+0000 7fef8dc1a740 1 Processor -- start 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 0x5604092b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef8dc1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a35c0 0x5604092b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef841a6930 con 0x5604092a2c00 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef841a7da0 con 0x5604092a1a30 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef8dc1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fef841a7000 con 0x5604092a35c0 2024-04-26T21:26:46.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef8dc1a740 10 client.0 _ll_register_callbacks cb 0x7fef7c00cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:26:46.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef83fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef83fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44670/0 (socket says 172.21.15.129:44670) 2024-04-26T21:26:46.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef83fff640 1 -- 172.21.15.129:0/3877017206 learned_addr learned my addr 172.21.15.129:0/3877017206 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:46.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef88bc1640 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 0x5604092b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef83fff640 1 -- 172.21.15.129:0/3877017206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a35c0 msgr2=0x5604092b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:46.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef83fff640 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a35c0 0x5604092b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef83fff640 1 -- 172.21.15.129:0/3877017206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 msgr2=0x5604092b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef83fff640 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 0x5604092b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef83fff640 1 -- 172.21.15.129:0/3877017206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604092b9df0 con 0x5604092a2c00 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef88bc1640 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 0x5604092b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef83fff640 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092b0d30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5604092a2b00 tx=0x7fef70019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c339991ae80229e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef63fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef63fff640 1 -- 172.21.15.129:0/3877017206 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7002e070 con 0x5604092a2c00 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef63fff640 1 -- 172.21.15.129:0/3877017206 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fef70028b50 con 0x5604092a2c00 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.313+0000 7fef8dc1a740 1 -- 172.21.15.129:0/3877017206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7c09c7c0 con 0x5604092a2c00 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef63fff640 1 -- 172.21.15.129:0/3877017206 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fef70039050 con 0x5604092a2c00 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef8dc1a740 10 client.16194 fetch_fsmap learned FSMap version 19 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef8dc1a740 1 -- 172.21.15.129:0/3877017206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5604092b3500 con 0x5604092a2c00 2024-04-26T21:26:46.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef63fff640 1 -- 172.21.15.129:0/3877017206 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c09c7c0 con 0x5604092a2c00 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef63fff640 1 -- 172.21.15.129:0/3877017206 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fef70028550 con 0x5604092a2c00 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef8dc1a740 10 client.16194 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef8dc1a740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef8dc1a740 20 client.16194 populate_metadata read hostname 'smithi129' 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56040931e320 0x560409320740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef8dc1a740 4 client.16194 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef8dc1a740 1 -- 172.21.15.129:0/3877017206 --> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56040920b800 con 0x56040931e320 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.314+0000 7fef88bc1640 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56040931e320 0x560409320740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef63fff640 10 client.16194 ms_handle_connect on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef88bc1640 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56040931e320 0x560409320740 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=153816c3d5dd3813 server_cookie=85ad20853bb1ca1b in_seq=0 out_seq=0 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef63fff640 1 -- 172.21.15.129:0/3877017206 <== mds.0 v2:172.21.15.150:6838/3984403456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56040920b800 con 0x56040931e320 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef63fff640 10 client.16194 handle_command_reply: tid=0 2024-04-26T21:26:46.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 client.16194 shutdown 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 -- 172.21.15.129:0/3877017206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 msgr2=0x5604092b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092b0d30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5604092a2b00 tx=0x7fef70019000 comp rx=0 tx=0).stop 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 -- 172.21.15.129:0/3877017206 shutdown_connections 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.150:6838/3984403456,v1:172.21.15.150:6839/3984403456] conn(0x56040931e320 0x560409320740 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604092a35c0 0x5604092b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604092a2c00 0x5604092b0d30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef63fff640 0 client.16194 ms_handle_reset on v2:172.21.15.150:6838/3984403456 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 --2- 172.21.15.129:0/3877017206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604092a1a30 0x5604092b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 -- 172.21.15.129:0/3877017206 >> 172.21.15.129:0/3877017206 conn(0x5604091e0910 msgr2=0x56040929e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 -- 172.21.15.129:0/3877017206 shutdown_connections 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 1 -- 172.21.15.129:0/3877017206 wait complete. 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.315+0000 7fef8dc1a740 20 client.16194 trim_cache size 0 max 16384 2024-04-26T21:26:46.325 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2024-04-26T21:26:46.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 -- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef740a3cc0 msgr2=0x7fef740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef740a3cc0 0x7fef740a6180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fef84198fe0 tx=0x7fef7000cbe0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 -- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84103420 msgr2=0x7fef8419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:46.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84103420 0x7fef8419f080 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00eaa0 tx=0x7fef7c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:26:46.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 -- 172.21.15.129:0/3987706763 shutdown_connections 2024-04-26T21:26:46.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fef740a3cc0 0x7fef740a6180 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84102a60 0x7fef84198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84103420 0x7fef8419f080 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 --2- 172.21.15.129:0/3987706763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fef84101870 0x7fef84197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:46.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 -- 172.21.15.129:0/3987706763 >> 172.21.15.129:0/3987706763 conn(0x7fef840fcde0 msgr2=0x7fef84105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:46.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 -- 172.21.15.129:0/3987706763 shutdown_connections 2024-04-26T21:26:46.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:46.316+0000 7fef8ae25640 1 -- 172.21.15.129:0/3987706763 wait complete. 2024-04-26T21:26:46.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:46 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3921751649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:26:46.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:46 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3921751649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:26:46.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:46 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3921751649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:26:47.329 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-26T21:26:47.329 INFO:tasks.fwd_scrub:done joining 2024-04-26T21:26:47.330 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T21:26:47.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:26:47.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 -- 172.21.15.129:0/633389035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c1034d0 msgr2=0x7f691c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 --2- 172.21.15.129:0/633389035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c1034d0 0x7f691c1058c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f690c009ca0 tx=0x7f690c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 -- 172.21.15.129:0/633389035 shutdown_connections 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 --2- 172.21.15.129:0/633389035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f691c105e00 0x7f691c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 --2- 172.21.15.129:0/633389035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f691c108730 0x7f691c10ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 --2- 172.21.15.129:0/633389035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c1034d0 0x7f691c1058c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 -- 172.21.15.129:0/633389035 >> 172.21.15.129:0/633389035 conn(0x7f691c0fce20 msgr2=0x7f691c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 -- 172.21.15.129:0/633389035 shutdown_connections 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 -- 172.21.15.129:0/633389035 wait complete. 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.450+0000 7f6921852640 1 Processor -- start 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f6921852640 1 -- start start 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f6921852640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f691c1034d0 0x7f691c102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f6921852640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f691c105e00 0x7f691c101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f6921852640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c108730 0x7f691c1019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:47.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f6921852640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10e4f0 con 0x7f691c108730 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f6921852640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10e370 con 0x7f691c105e00 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f6921852640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10e670 con 0x7f691c1034d0 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c108730 0x7f691c1019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c108730 0x7f691c1019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44692/0 (socket says 172.21.15.129:44692) 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691b7fe640 1 -- 172.21.15.129:0/713703844 learned_addr learned my addr 172.21.15.129:0/713703844 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691affd640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f691c1034d0 0x7f691c102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691a7fc640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f691c105e00 0x7f691c101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691b7fe640 1 -- 172.21.15.129:0/713703844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f691c1034d0 msgr2=0x7f691c102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691b7fe640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f691c1034d0 0x7f691c102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691b7fe640 1 -- 172.21.15.129:0/713703844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f691c105e00 msgr2=0x7f691c101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691b7fe640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f691c105e00 0x7f691c101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691b7fe640 1 -- 172.21.15.129:0/713703844 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c102250 con 0x7f691c108730 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.451+0000 7f691b7fe640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c108730 0x7f691c1019c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f691400d750 tx=0x7f691400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.452+0000 7f6920850640 1 -- 172.21.15.129:0/713703844 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914017070 con 0x7f691c108730 2024-04-26T21:26:47.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.452+0000 7f6920850640 1 -- 172.21.15.129:0/713703844 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69140054d0 con 0x7f691c108730 2024-04-26T21:26:47.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.452+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1a2d50 con 0x7f691c108730 2024-04-26T21:26:47.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.452+0000 7f6920850640 1 -- 172.21.15.129:0/713703844 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914004c60 con 0x7f691c108730 2024-04-26T21:26:47.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.452+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1a31e0 con 0x7f691c108730 2024-04-26T21:26:47.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.453+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c0fe910 con 0x7f691c108730 2024-04-26T21:26:47.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.454+0000 7f6920850640 1 -- 172.21.15.129:0/713703844 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6914009bd0 con 0x7f691c108730 2024-04-26T21:26:47.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.454+0000 7f6920850640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f69040a3c40 0x7f69040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:47.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.455+0000 7f691affd640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f69040a3c40 0x7f69040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:47.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.455+0000 7f6920850640 1 -- 172.21.15.129:0/713703844 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f69140d0300 con 0x7f691c108730 2024-04-26T21:26:47.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.455+0000 7f691affd640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f69040a3c40 0x7f69040a6100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f690c005e30 tx=0x7f690c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:47.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.458+0000 7f6920850640 1 -- 172.21.15.129:0/713703844 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f691409cb70 con 0x7f691c108730 2024-04-26T21:26:47.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.638+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f691c063530 con 0x7f691c108730 2024-04-26T21:26:47.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.640+0000 7f6920850640 1 -- 172.21.15.129:0/713703844 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7f691409c470 con 0x7f691c108730 2024-04-26T21:26:47.641 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:26:47.641 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:26:47.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f69040a3c40 msgr2=0x7f69040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f69040a3c40 0x7f69040a6100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f690c005e30 tx=0x7f690c005dc0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c108730 msgr2=0x7f691c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c108730 0x7f691c1019c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f691400d750 tx=0x7f691400dc10 comp rx=0 tx=0).stop 2024-04-26T21:26:47.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 shutdown_connections 2024-04-26T21:26:47.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f69040a3c40 0x7f69040a6100 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f691c105e00 0x7f691c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f691c108730 0x7f691c1019c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 --2- 172.21.15.129:0/713703844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f691c1034d0 0x7f691c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 >> 172.21.15.129:0/713703844 conn(0x7f691c0fce20 msgr2=0x7f691c103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:47.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 shutdown_connections 2024-04-26T21:26:47.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.642+0000 7f6921852640 1 -- 172.21.15.129:0/713703844 wait complete. 2024-04-26T21:26:47.653 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:47 smithi129 ceph-mon[29237]: pgmap v614: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 342 KiB/s wr, 62 op/s 2024-04-26T21:26:47.655 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:16:01.621500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:16:02.592849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:26:47.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:26:47.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.754+0000 7fa35ebaf640 1 -- 172.21.15.129:0/3146706304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358102e90 msgr2=0x7fa3581032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.754+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/3146706304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358102e90 0x7fa3581032b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009900 tx=0x7fa34c02a410 comp rx=0 tx=0).stop 2024-04-26T21:26:47.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.754+0000 7fa35ebaf640 1 -- 172.21.15.129:0/3146706304 shutdown_connections 2024-04-26T21:26:47.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.754+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/3146706304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa358101ae0 0x7fa358101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.754+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/3146706304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa3581024a0 0x7fa358109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.754+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/3146706304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358102e90 0x7fa3581032b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.754+0000 7fa35ebaf640 1 -- 172.21.15.129:0/3146706304 >> 172.21.15.129:0/3146706304 conn(0x7fa3580fc570 msgr2=0x7fa3580fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.754+0000 7fa35ebaf640 1 -- 172.21.15.129:0/3146706304 shutdown_connections 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.754+0000 7fa35ebaf640 1 -- 172.21.15.129:0/3146706304 wait complete. 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35ebaf640 1 Processor -- start 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35ebaf640 1 -- start start 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35ebaf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358101ae0 0x7fa358101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35ebaf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581024a0 0x7fa35819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35ebaf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa358102e90 0x7fa35819ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35ebaf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa35810d3c0 con 0x7fa358101ae0 2024-04-26T21:26:47.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35ebaf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa35810d240 con 0x7fa3581024a0 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35ebaf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa35810d540 con 0x7fa358102e90 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35c94b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358101ae0 0x7fa358101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35c94b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358101ae0 0x7fa358101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44714/0 (socket says 172.21.15.129:44714) 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35c94b640 1 -- 172.21.15.129:0/329488938 learned_addr learned my addr 172.21.15.129:0/329488938 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35d14c640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa358102e90 0x7fa35819ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa357fff640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581024a0 0x7fa35819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35c94b640 1 -- 172.21.15.129:0/329488938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa358102e90 msgr2=0x7fa35819ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35c94b640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa358102e90 0x7fa35819ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35c94b640 1 -- 172.21.15.129:0/329488938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581024a0 msgr2=0x7fa35819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35c94b640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581024a0 0x7fa35819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35c94b640 1 -- 172.21.15.129:0/329488938 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3581a0890 con 0x7fa358101ae0 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35d14c640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa358102e90 0x7fa35819ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:26:47.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa357fff640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581024a0 0x7fa35819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:26:47.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35c94b640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358101ae0 0x7fa358101540 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa34c006010 tx=0x7fa34c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:47.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa355ffb640 1 -- 172.21.15.129:0/329488938 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c03f030 con 0x7fa358101ae0 2024-04-26T21:26:47.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa355ffb640 1 -- 172.21.15.129:0/329488938 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c003cb0 con 0x7fa358101ae0 2024-04-26T21:26:47.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3581a0ac0 con 0x7fa358101ae0 2024-04-26T21:26:47.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.755+0000 7fa355ffb640 1 -- 172.21.15.129:0/329488938 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c035070 con 0x7fa358101ae0 2024-04-26T21:26:47.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.756+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3581a0fb0 con 0x7fa358101ae0 2024-04-26T21:26:47.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.757+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa358108ce0 con 0x7fa358101ae0 2024-04-26T21:26:47.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.758+0000 7fa355ffb640 1 -- 172.21.15.129:0/329488938 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa34c042030 con 0x7fa358101ae0 2024-04-26T21:26:47.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.759+0000 7fa355ffb640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa3480a3c40 0x7fa3480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:47.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.759+0000 7fa357fff640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa3480a3c40 0x7fa3480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:47.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.759+0000 7fa355ffb640 1 -- 172.21.15.129:0/329488938 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa34c0f0c90 con 0x7fa358101ae0 2024-04-26T21:26:47.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.759+0000 7fa357fff640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa3480a3c40 0x7fa3480a6100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa3581012b0 tx=0x7fa34400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:47.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.763+0000 7fa355ffb640 1 -- 172.21.15.129:0/329488938 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa34c0f6020 con 0x7fa358101ae0 2024-04-26T21:26:47.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:47 smithi150 ceph-mon[32290]: pgmap v614: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 342 KiB/s wr, 62 op/s 2024-04-26T21:26:47.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.939+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa35802dd40 con 0x7fa358101ae0 2024-04-26T21:26:47.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.939+0000 7fa355ffb640 1 -- 172.21.15.129:0/329488938 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fa34c0bd500 con 0x7fa358101ae0 2024-04-26T21:26:47.940 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:26:47.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa3480a3c40 msgr2=0x7fa3480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa3480a3c40 0x7fa3480a6100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa3581012b0 tx=0x7fa34400a560 comp rx=0 tx=0).stop 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358101ae0 msgr2=0x7fa358101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358101ae0 0x7fa358101540 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa34c006010 tx=0x7fa34c0042f0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 shutdown_connections 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fa3480a3c40 0x7fa3480a6100 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581024a0 0x7fa35819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa358102e90 0x7fa35819ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 --2- 172.21.15.129:0/329488938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa358101ae0 0x7fa358101540 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 >> 172.21.15.129:0/329488938 conn(0x7fa3580fc570 msgr2=0x7fa358105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 shutdown_connections 2024-04-26T21:26:47.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:47.941+0000 7fa35ebaf640 1 -- 172.21.15.129:0/329488938 wait complete. 2024-04-26T21:26:47.954 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:26:47.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:47 smithi171 ceph-mon[33478]: pgmap v614: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 342 KiB/s wr, 62 op/s 2024-04-26T21:26:48.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 -- 172.21.15.129:0/690253116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4101890 msgr2=0x7fd2d4101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/690253116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4101890 0x7fd2d4101cb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009900 tx=0x7fd2c402a410 comp rx=0 tx=0).stop 2024-04-26T21:26:48.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 -- 172.21.15.129:0/690253116 shutdown_connections 2024-04-26T21:26:48.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/690253116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d4102a80 0x7fd2d4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/690253116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2d4103440 0x7fd2d4109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/690253116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4101890 0x7fd2d4101cb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 -- 172.21.15.129:0/690253116 >> 172.21.15.129:0/690253116 conn(0x7fd2d40fce00 msgr2=0x7fd2d40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 -- 172.21.15.129:0/690253116 shutdown_connections 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 -- 172.21.15.129:0/690253116 wait complete. 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 Processor -- start 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.054+0000 7fd2d9c99640 1 -- start start 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d9c99640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2d4101890 0x7fd2d4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d9c99640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d4102a80 0x7fd2d4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d9c99640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4103440 0x7fd2d41a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d9c99640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d410e4e0 con 0x7fd2d4103440 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d9c99640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d410e360 con 0x7fd2d4102a80 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d9c99640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d410e660 con 0x7fd2d4101890 2024-04-26T21:26:48.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4103440 0x7fd2d41a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4103440 0x7fd2d41a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44744/0 (socket says 172.21.15.129:44744) 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d3fff640 1 -- 172.21.15.129:0/2815045573 learned_addr learned my addr 172.21.15.129:0/2815045573 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2cbfff640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d4102a80 0x7fd2d4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d37fe640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2d4101890 0x7fd2d4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d3fff640 1 -- 172.21.15.129:0/2815045573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2d4101890 msgr2=0x7fd2d4198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d3fff640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2d4101890 0x7fd2d4198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d3fff640 1 -- 172.21.15.129:0/2815045573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d4102a80 msgr2=0x7fd2d4199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d3fff640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d4102a80 0x7fd2d4199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d3fff640 1 -- 172.21.15.129:0/2815045573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d41a0850 con 0x7fd2d4103440 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d37fe640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2d4101890 0x7fd2d4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d3fff640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4103440 0x7fd2d41a01b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00cad0 tx=0x7fd2cc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d17fa640 1 -- 172.21.15.129:0/2815045573 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc00ae20 con 0x7fd2d4103440 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d41a0b40 con 0x7fd2d4103440 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d17fa640 1 -- 172.21.15.129:0/2815045573 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2cc005020 con 0x7fd2d4103440 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d17fa640 1 -- 172.21.15.129:0/2815045573 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc00f6b0 con 0x7fd2d4103440 2024-04-26T21:26:48.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.055+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d41a1090 con 0x7fd2d4103440 2024-04-26T21:26:48.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.056+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d4108cd0 con 0x7fd2d4103440 2024-04-26T21:26:48.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.057+0000 7fd2d17fa640 1 -- 172.21.15.129:0/2815045573 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd2cc0051d0 con 0x7fd2d4103440 2024-04-26T21:26:48.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.058+0000 7fd2d17fa640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd2c00a3c40 0x7fd2c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.058+0000 7fd2d37fe640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd2c00a3c40 0x7fd2c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.058+0000 7fd2d17fa640 1 -- 172.21.15.129:0/2815045573 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd2cc0d0aa0 con 0x7fd2d4103440 2024-04-26T21:26:48.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.059+0000 7fd2d37fe640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd2c00a3c40 0x7fd2c00a6100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4006010 tx=0x7fd2c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:48.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.062+0000 7fd2d17fa640 1 -- 172.21.15.129:0/2815045573 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd2cc014030 con 0x7fd2d4103440 2024-04-26T21:26:48.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.238+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd2d402dc60 con 0x7fd2d4103440 2024-04-26T21:26:48.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.238+0000 7fd2d17fa640 1 -- 172.21.15.129:0/2815045573 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fd2cc09d310 con 0x7fd2d4103440 2024-04-26T21:26:48.239 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-26T21:26:48.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd2c00a3c40 msgr2=0x7fd2c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd2c00a3c40 0x7fd2c00a6100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4006010 tx=0x7fd2c4000bc0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4103440 msgr2=0x7fd2d41a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4103440 0x7fd2d41a01b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00cad0 tx=0x7fd2cc00cf90 comp rx=0 tx=0).stop 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 shutdown_connections 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd2c00a3c40 0x7fd2c00a6100 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d4102a80 0x7fd2d4199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2d4103440 0x7fd2d41a01b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 --2- 172.21.15.129:0/2815045573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2d4101890 0x7fd2d4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 >> 172.21.15.129:0/2815045573 conn(0x7fd2d40fce00 msgr2=0x7fd2d4105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 shutdown_connections 2024-04-26T21:26:48.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.240+0000 7fd2d9c99640 1 -- 172.21.15.129:0/2815045573 wait complete. 2024-04-26T21:26:48.253 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:26:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 -- 172.21.15.129:0/2856436957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce4103c10 msgr2=0x7f4ce4104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 --2- 172.21.15.129:0/2856436957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce4103c10 0x7f4ce4104090 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009930 tx=0x7f4cd402a420 comp rx=0 tx=0).stop 2024-04-26T21:26:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 -- 172.21.15.129:0/2856436957 shutdown_connections 2024-04-26T21:26:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 --2- 172.21.15.129:0/2856436957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ce41045d0 0x7f4ce410ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 --2- 172.21.15.129:0/2856436957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce4103c10 0x7f4ce4104090 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 --2- 172.21.15.129:0/2856436957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ce4100c40 0x7f4ce4101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 -- 172.21.15.129:0/2856436957 >> 172.21.15.129:0/2856436957 conn(0x7f4ce40fc570 msgr2=0x7f4ce40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 -- 172.21.15.129:0/2856436957 shutdown_connections 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 -- 172.21.15.129:0/2856436957 wait complete. 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 Processor -- start 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.353+0000 7f4cebd46640 1 -- start start 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cebd46640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ce4100c40 0x7f4ce4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cebd46640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ce4103c10 0x7f4ce4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cebd46640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce41045d0 0x7f4ce41a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cebd46640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce410e4d0 con 0x7f4ce41045d0 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cebd46640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce410e350 con 0x7f4ce4103c10 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cebd46640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce410e650 con 0x7f4ce4100c40 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cea2e3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce41045d0 0x7f4ce41a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cea2e3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce41045d0 0x7f4ce41a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44768/0 (socket says 172.21.15.129:44768) 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cea2e3640 1 -- 172.21.15.129:0/1433943672 learned_addr learned my addr 172.21.15.129:0/1433943672 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4ce9ae2640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ce4100c40 0x7f4ce4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4ce92e1640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ce4103c10 0x7f4ce4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cea2e3640 1 -- 172.21.15.129:0/1433943672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ce4100c40 msgr2=0x7f4ce4198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cea2e3640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ce4100c40 0x7f4ce4198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cea2e3640 1 -- 172.21.15.129:0/1433943672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ce4103c10 msgr2=0x7f4ce4199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cea2e3640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ce4103c10 0x7f4ce4199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cea2e3640 1 -- 172.21.15.129:0/1433943672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce41a0850 con 0x7f4ce41045d0 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cea2e3640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce41045d0 0x7f4ce41a01b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00d750 tx=0x7f4cdc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cdaffd640 1 -- 172.21.15.129:0/1433943672 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc017070 con 0x7f4ce41045d0 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cdaffd640 1 -- 172.21.15.129:0/1433943672 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cdc0054d0 con 0x7f4ce41045d0 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce41a0b40 con 0x7f4ce41045d0 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cdaffd640 1 -- 172.21.15.129:0/1433943672 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc004c60 con 0x7f4ce41045d0 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.354+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce41a1090 con 0x7f4ce41045d0 2024-04-26T21:26:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.355+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce4109e10 con 0x7f4ce41045d0 2024-04-26T21:26:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.356+0000 7f4cdaffd640 1 -- 172.21.15.129:0/1433943672 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4cdc009bd0 con 0x7f4ce41045d0 2024-04-26T21:26:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.357+0000 7f4cdaffd640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f4cd00a3c40 0x7f4cd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.357+0000 7f4ce9ae2640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f4cd00a3c40 0x7f4cd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.357+0000 7f4cdaffd640 1 -- 172.21.15.129:0/1433943672 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f4cdc0d02e0 con 0x7f4ce41045d0 2024-04-26T21:26:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.357+0000 7f4ce9ae2640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f4cd00a3c40 0x7f4cd00a6100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ce419a110 tx=0x7f4ce0009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:48.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.361+0000 7f4cdaffd640 1 -- 172.21.15.129:0/1433943672 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4cdc09c500 con 0x7f4ce41045d0 2024-04-26T21:26:48.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.539+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ce4063530 con 0x7f4ce41045d0 2024-04-26T21:26:48.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.539+0000 7f4cdaffd640 1 -- 172.21.15.129:0/1433943672 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f4cdc0a0800 con 0x7f4ce41045d0 2024-04-26T21:26:48.540 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-26T21:26:48.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f4cd00a3c40 msgr2=0x7f4cd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f4cd00a3c40 0x7f4cd00a6100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ce419a110 tx=0x7f4ce0009670 comp rx=0 tx=0).stop 2024-04-26T21:26:48.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce41045d0 msgr2=0x7f4ce41a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce41045d0 0x7f4ce41a01b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00d750 tx=0x7f4cdc00dc10 comp rx=0 tx=0).stop 2024-04-26T21:26:48.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 shutdown_connections 2024-04-26T21:26:48.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f4cd00a3c40 0x7f4cd00a6100 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ce4103c10 0x7f4ce4199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ce41045d0 0x7f4ce41a01b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 --2- 172.21.15.129:0/1433943672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ce4100c40 0x7f4ce4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 >> 172.21.15.129:0/1433943672 conn(0x7f4ce40fc570 msgr2=0x7f4ce4106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:48.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 shutdown_connections 2024-04-26T21:26:48.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.541+0000 7f4cebd46640 1 -- 172.21.15.129:0/1433943672 wait complete. 2024-04-26T21:26:48.553 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.650+0000 7f10d3d19640 1 -- 172.21.15.129:0/2098513135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 msgr2=0x7f10cc109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.650+0000 7f10d3d19640 1 --2- 172.21.15.129:0/2098513135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 0x7f10cc109cc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009ca0 tx=0x7f10c402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.650+0000 7f10d3d19640 1 -- 172.21.15.129:0/2098513135 shutdown_connections 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.650+0000 7f10d3d19640 1 --2- 172.21.15.129:0/2098513135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10cc102a70 0x7f10cc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.650+0000 7f10d3d19640 1 --2- 172.21.15.129:0/2098513135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 0x7f10cc109cc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.650+0000 7f10d3d19640 1 --2- 172.21.15.129:0/2098513135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10cc101880 0x7f10cc101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.650+0000 7f10d3d19640 1 -- 172.21.15.129:0/2098513135 >> 172.21.15.129:0/2098513135 conn(0x7f10cc0fce10 msgr2=0x7f10cc0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.650+0000 7f10d3d19640 1 -- 172.21.15.129:0/2098513135 shutdown_connections 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.650+0000 7f10d3d19640 1 -- 172.21.15.129:0/2098513135 wait complete. 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d3d19640 1 Processor -- start 2024-04-26T21:26:48.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d3d19640 1 -- start start 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d3d19640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10cc101880 0x7f10cc1058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d3d19640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10cc102a70 0x7f10cc103f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d3d19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 0x7f10cc104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d3d19640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc10d390 con 0x7f10cc103430 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d3d19640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc10d210 con 0x7f10cc102a70 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d3d19640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc10d510 con 0x7f10cc101880 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d22b6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 0x7f10cc104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d22b6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 0x7f10cc104460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44784/0 (socket says 172.21.15.129:44784) 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d22b6640 1 -- 172.21.15.129:0/409150859 learned_addr learned my addr 172.21.15.129:0/409150859 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d12b4640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10cc102a70 0x7f10cc103f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d1ab5640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10cc101880 0x7f10cc1058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d22b6640 1 -- 172.21.15.129:0/409150859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10cc101880 msgr2=0x7f10cc1058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d22b6640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10cc101880 0x7f10cc1058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d22b6640 1 -- 172.21.15.129:0/409150859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10cc102a70 msgr2=0x7f10cc103f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d22b6640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10cc102a70 0x7f10cc103f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d22b6640 1 -- 172.21.15.129:0/409150859 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc104cf0 con 0x7f10cc103430 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d1ab5640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10cc101880 0x7f10cc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d12b4640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10cc102a70 0x7f10cc103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.651+0000 7f10d22b6640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 0x7f10cc104460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f10c40097b0 tx=0x7f10c4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.652+0000 7f10c2ffd640 1 -- 172.21.15.129:0/409150859 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4004240 con 0x7f10cc103430 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.652+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc1a2bd0 con 0x7f10cc103430 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.652+0000 7f10c2ffd640 1 -- 172.21.15.129:0/409150859 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c40317c0 con 0x7f10cc103430 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.652+0000 7f10c2ffd640 1 -- 172.21.15.129:0/409150859 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c402fd80 con 0x7f10cc103430 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.652+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc1a3120 con 0x7f10cc103430 2024-04-26T21:26:48.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.653+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc1a32f0 con 0x7f10cc103430 2024-04-26T21:26:48.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.654+0000 7f10c2ffd640 1 -- 172.21.15.129:0/409150859 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f10c40043e0 con 0x7f10cc103430 2024-04-26T21:26:48.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.654+0000 7f10c2ffd640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f10b80a3c40 0x7f10b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:48.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.654+0000 7f10d1ab5640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f10b80a3c40 0x7f10b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:48.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.654+0000 7f10c2ffd640 1 -- 172.21.15.129:0/409150859 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f10c40f1f40 con 0x7f10cc103430 2024-04-26T21:26:48.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.655+0000 7f10d1ab5640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f10b80a3c40 0x7f10b80a6100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f10cc1028d0 tx=0x7f10c800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:48.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.658+0000 7f10c2ffd640 1 -- 172.21.15.129:0/409150859 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f10c40be8c0 con 0x7f10cc103430 2024-04-26T21:26:48.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:48 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/713703844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:26:48.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:48 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/329488938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:26:48.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:48 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/2815045573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:26:48.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:48 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1433943672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:26:48.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.882+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f10cc1a3680 con 0x7f10cc103430 2024-04-26T21:26:48.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.890+0000 7f10c2ffd640 1 -- 172.21.15.129:0/409150859 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v17) ==== 81+0+745785 (secure 0 0 0) 0x7f10c40be1c0 con 0x7f10cc103430 2024-04-26T21:26:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:26:48.907 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:26:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 17, 2024-04-26T21:26:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "active_gid": 14152, 2024-04-26T21:26:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "active_name": "x", 2024-04-26T21:26:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "active_addrs": { 2024-04-26T21:26:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-04-26T21:26:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6800", 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3856000307 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6801", 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3856000307 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "active_addr": "172.21.15.129:6801/3856000307", 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "active_change": "2024-04-26T21:08:55.174206+0000", 2024-04-26T21:26:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "available": true, 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "standbys": [ 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "gid": 24109, 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "z", 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_features": 4540701547738038271, 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "available_modules": [ 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "alerts", 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_destination": { 2024-04-26T21:26:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_destination", 2024-04-26T21:26:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:26:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_from_name": { 2024-04-26T21:26:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_from_name", 2024-04-26T21:26:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "Ceph", 2024-04-26T21:26:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email From: name", 2024-04-26T21:26:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_host": { 2024-04-26T21:26:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_host", 2024-04-26T21:26:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP server", 2024-04-26T21:26:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_password": { 2024-04-26T21:26:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_password", 2024-04-26T21:26:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Password to authenticate with", 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_port": { 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_port", 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "465", 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP port", 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_sender": { 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_sender", 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_ssl": { 2024-04-26T21:26:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_ssl", 2024-04-26T21:26:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_user": { 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_user", 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "User to authenticate as", 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "balancer", 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "active": { 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "active", 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_time": { 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_time", 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0000", 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_weekday": { 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_weekday", 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_max_iterations": { 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "25", 2024-04-26T21:26:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "250", 2024-04-26T21:26:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:26:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_metrics": { 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_metrics", 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_step": { 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_step", 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.5", 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0.001", 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "0.999", 2024-04-26T21:26:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "end_time": { 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_time", 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2359", 2024-04-26T21:26:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "end_weekday": { 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_weekday", 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "min_score": { 2024-04-26T21:26:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_score", 2024-04-26T21:26:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "upmap", 2024-04-26T21:26:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-compat", 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "none", 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "read", 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap", 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap-read" 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Balancer mode", 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_ids": { 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_ids", 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_deviation": { 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_deviation", 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_optimizations": { 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:26:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:26:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm", 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_down_multiplier": { 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_down_multiplier", 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:26:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_refresh_rate": { 2024-04-26T21:26:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_refresh_rate", 2024-04-26T21:26:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "20", 2024-04-26T21:26:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_starting_port": { 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_starting_port", 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.959 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4721", 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_ptrace": { 2024-04-26T21:26:48.960 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_ptrace", 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:26:48.961 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.962 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.962 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_interval": { 2024-04-26T21:26:48.962 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_interval", 2024-04-26T21:26:48.962 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:48.962 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.963 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.963 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:48.963 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.963 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.963 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.963 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:26:48.963 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.963 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.963 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.964 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:26:48.964 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:26:48.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:48.964 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.964 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.7", 2024-04-26T21:26:48.964 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.964 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm_log_destination": { 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm_log_destination", 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.966 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:48.966 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.966 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.966 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:48.966 INFO:teuthology.orchestra.run.smithi129.stdout: "file", 2024-04-26T21:26:48.966 INFO:teuthology.orchestra.run.smithi129.stdout: "file,syslog", 2024-04-26T21:26:48.966 INFO:teuthology.orchestra.run.smithi129.stdout: "syslog" 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "cgroups_split": { 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cgroups_split", 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.967 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "config_checks_enabled": { 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_checks_enabled", 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.968 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "config_dashboard": { 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_dashboard", 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:26:48.970 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_alertmanager": { 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_alertmanager", 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.971 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_base": { 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_base", 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.972 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_elasticsearch": { 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "elasticsearch container image", 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_grafana": { 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_grafana", 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_haproxy": { 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_haproxy", 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "HAproxy container image", 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_agent": { 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_collector": { 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_query": { 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger query container image", 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_keepalived": { 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_keepalived", 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Keepalived container image", 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_loki": { 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_loki", 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Loki container image", 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_node_exporter": { 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_node_exporter", 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_nvmeof": { 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_nvmeof", 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Nvme-of container image", 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_prometheus": { 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_prometheus", 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_promtail": { 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_promtail", 2024-04-26T21:26:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Promtail container image", 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_samba": { 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_samba", 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_snmp_gateway": { 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "container_init": { 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_init", 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "daemon_cache_timeout": { 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "900", 2024-04-26T21:26:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:26:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "default_registry": { 2024-04-26T21:26:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_registry", 2024-04-26T21:26:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io", 2024-04-26T21:26:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "device_cache_timeout": { 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_cache_timeout", 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1800", 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "device_enhanced_scan": { 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_enhanced_scan", 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:26:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "facts_cache_timeout": { 2024-04-26T21:26:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "facts_cache_timeout", 2024-04-26T21:26:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "grafana_dashboards_path": { 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "host_check_interval": { 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "host_check_interval", 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "hw_monitoring": { 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hw_monitoring", 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:26:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "inventory_list_all": { 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "inventory_list_all", 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "log_refresh_metadata": { 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_refresh_metadata", 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "*", 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "max_count_per_host": { 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_count_per_host", 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd_draining_count": { 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_osd_draining_count", 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "migration_current": { 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "migration_current", 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "internal - do not modify", 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "root", 2024-04-26T21:26:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm-package", 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "root" 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "oob_default_addr": { 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "oob_default_addr", 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "169.254.1.1", 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "prometheus_alerts_path": { 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_insecure": { 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_insecure", 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_password": { 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_password", 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_url": { 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_url", 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_username": { 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_username", 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "secure_monitoring_stack": { 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "service_discovery_port": { 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "service_discovery_port", 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8765", 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "ssh_config_file": { 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssh_config_file", 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "use_agent": { 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_agent", 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.024 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "use_repo_digest": { 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_repo_digest", 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.025 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_failed_host_check": { 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.026 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_daemons": { 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.027 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_hosts": { 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.028 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crash", 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.030 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.031 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.032 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "retain_interval": { 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retain_interval", 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "31536000", 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_recent_interval": { 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_recent_interval", 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1209600", 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dashboard", 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.037 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.038 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.039 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.040 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.041 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.042 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.044 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.045 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.046 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.047 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.049 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.050 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.052 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_URL": { 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:26:49.053 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:26:49.054 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:26:49.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.055 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.055 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.055 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.055 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.055 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.055 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.057 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.057 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.057 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.057 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:26:49.057 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:26:49.057 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.057 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.058 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.060 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "[]", 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.061 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "{}", 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.062 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.063 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.064 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.065 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.066 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.067 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.068 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.069 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.070 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.071 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.072 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.073 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8", 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.074 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "45", 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.075 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.078 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.079 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.080 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "cross_origin_url": { 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cross_origin_url", 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.084 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "crt_file": { 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crt_file", 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.085 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "debug": { 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "debug", 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.086 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "jwt_token_ttl": { 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "jwt_token_ttl", 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "28800", 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.087 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.089 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.090 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.091 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.092 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.093 INFO:teuthology.orchestra.run.smithi129.stdout: "motd": { 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "motd", 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "The message of the day", 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.094 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.095 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8080", 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl_server_port": { 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl_server_port", 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8443", 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "redirect", 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect" 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.100 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.101 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "url_prefix": { 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url_prefix", 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "devicehealth", 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_monitoring": { 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_monitoring", 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "monitor device health metrics", 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.104 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.107 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.108 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "mark_out_threshold": { 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mark_out_threshold", 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2419200", 2024-04-26T21:26:49.109 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": { 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_name", 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.110 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "device_health_metrics", 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "retention_period": { 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retention_period", 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.111 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15552000", 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_frequency": { 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_frequency", 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.112 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "self_heal": { 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "self_heal", 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.113 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.114 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_threshold": { 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_threshold", 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.115 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7257600", 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "diskprediction_local", 2024-04-26T21:26:49.116 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.117 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.118 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.119 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "predict_interval": { 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predict_interval", 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "predictor_model": { 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predictor_model", 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "prophetstor", 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "influx", 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": false, 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "batch_size": { 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "batch_size", 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5000", 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "database": { 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "database", 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "ceph", 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.126 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": { 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hostname", 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.127 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "30", 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "5", 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.129 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.130 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.131 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.132 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.133 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "password": { 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "password", 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.134 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "port": { 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "port", 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8086", 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server port", 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.135 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "false", 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.136 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "threads": { 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threads", 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "32", 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.137 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "username": { 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "username", 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.138 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "verify_ssl": { 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "verify_ssl", 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "true", 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.139 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "insights", 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.140 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.143 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.144 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "iostat", 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.145 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.146 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.147 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.148 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.149 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "k8sevents", 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_event_retention_days": { 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7", 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "config_check_secs": { 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_check_secs", 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "10", 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:26:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "localpool", 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "failure_domain": { 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "failure_domain", 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "host", 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": { 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_size", 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "num_rep": { 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "num_rep", 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3", 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": { 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_num", 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "128", 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "prefix": { 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prefix", 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "subtree": { 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "subtree", 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "rack", 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:26:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mds_autoscaler", 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirroring", 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "nfs", 2024-04-26T21:26:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "fail_fs": { 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "fail_fs", 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator": { 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm", 2024-04-26T21:26:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "rook", 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "test_orchestrator" 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Orchestrator backend", 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_perf_query", 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_support", 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_autoscaler", 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "threshold": { 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threshold", 2024-04-26T21:26:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1.0", 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "scaling threshold", 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "progress", 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_pg_recovery_event": { 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "max_completed_events": { 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_completed_events", 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "50", 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:26:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus", 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "cache": { 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cache", 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "exclude_perf_counters": { 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "exclude_perf_counters", 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools": { 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools", 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "300", 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_interval": { 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_interval", 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15.0", 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "9283", 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "stale_cache_strategy": { 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stale_cache_strategy", 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "log", 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "default", 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "default", 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "error" 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_support", 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_snap_create": { 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "trash_purge_schedule": { 2024-04-26T21:26:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "trash_purge_schedule", 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "restful", 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_auth": { 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_auth", 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.248 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.249 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.250 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.251 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.252 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rgw", 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.253 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.254 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.255 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.256 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.257 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rook", 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.258 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.259 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.260 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.261 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.262 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "storage_class": { 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "storage_class", 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.263 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "local", 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "selftest", 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.264 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.266 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.267 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi129.stdout: "roption1": { 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption1", 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "roption2": { 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption2", 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "xyz", 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption1": { 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption1", 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption2": { 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption2", 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption3": { 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption3", 2024-04-26T21:26:49.273 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption4": { 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption4", 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.274 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption5": { 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption5", 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.275 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption6": { 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption6", 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.276 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption7": { 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption7", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:49.277 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "42", 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "testkey": { 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testkey", 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.278 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "testlkey": { 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testlkey", 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.279 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "testnewline": { 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testnewline", 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.280 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snap_schedule", 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_m_granularity": { 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_m_granularity", 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.281 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "dump_on_update": { 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dump_on_update", 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.282 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.283 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.284 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.286 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.287 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stats", 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.288 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.291 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.293 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.294 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "status", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.295 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.297 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.298 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.299 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telegraf", 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "address": { 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "address", 2024-04-26T21:26:49.300 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.301 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15", 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.302 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.303 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.304 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.305 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.306 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.307 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telemetry", 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_basic": { 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_basic", 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:26:49.308 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_crash": { 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_crash", 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.309 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_device": { 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_device", 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.310 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_ident": { 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_ident", 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.311 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_perf": { 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_perf", 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "contact": { 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "contact", 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.313 INFO:teuthology.orchestra.run.smithi129.stdout: "description": { 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "description", 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.314 INFO:teuthology.orchestra.run.smithi129.stdout: "device_url": { 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_url", 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.315 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.316 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "24", 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "8", 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "last_opt_revision": { 2024-04-26T21:26:49.317 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "last_opt_revision", 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1", 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard": { 2024-04-26T21:26:49.318 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard", 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard_description": { 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard_description", 2024-04-26T21:26:49.319 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.320 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.321 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.322 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.323 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.324 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "organization": { 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "organization", 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.325 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "proxy": { 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "proxy", 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.326 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "url": { 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url", 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.327 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "test_orchestrator", 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.329 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.330 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.331 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.332 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "volumes", 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.333 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.334 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.335 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.336 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.337 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.338 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_clones": { 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_clones", 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4", 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "periodic_async_work": { 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "periodic_async_work", 2024-04-26T21:26:49.339 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Periodically check for async work", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_delay": { 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.340 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.341 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix", 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.342 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "discovery_interval": { 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "discovery_interval", 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "100", 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "identifier": { 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "identifier", 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.344 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.345 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.346 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.347 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.348 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.349 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_host": { 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_host", 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.350 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_port": { 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_port", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10051", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.351 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_sender": { 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_sender", 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.352 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: "gid": 24110, 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "y", 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_features": 4540701547738038271, 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: "available_modules": [ 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.353 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "alerts", 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:26:49.354 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.355 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.356 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.357 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_destination": { 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_destination", 2024-04-26T21:26:49.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_from_name": { 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_from_name", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.360 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "Ceph", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email From: name", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_host": { 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_host", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.361 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP server", 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_password": { 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_password", 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.362 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Password to authenticate with", 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_port": { 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_port", 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "465", 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.363 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP port", 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_sender": { 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_sender", 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.364 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_ssl": { 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_ssl", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.365 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_user": { 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_user", 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "User to authenticate as", 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.366 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "balancer", 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "active": { 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "active", 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.367 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_time": { 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_time", 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0000", 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.368 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_weekday": { 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_weekday", 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:26:49.369 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_max_iterations": { 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "25", 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "250", 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:26:49.370 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_metrics": { 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_metrics", 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:26:49.371 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_step": { 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_step", 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.5", 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0.001", 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "0.999", 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "end_time": { 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_time", 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2359", 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "end_weekday": { 2024-04-26T21:26:49.373 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_weekday", 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.374 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.375 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.378 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "min_score": { 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_score", 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.379 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.380 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "upmap", 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-compat", 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "none", 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "read", 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap", 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap-read" 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Balancer mode", 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.381 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_ids": { 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_ids", 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.382 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_deviation": { 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_deviation", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.383 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_optimizations": { 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.384 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm", 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.385 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_down_multiplier": { 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_down_multiplier", 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.386 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_refresh_rate": { 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_refresh_rate", 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "20", 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.387 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_starting_port": { 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_starting_port", 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4721", 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.388 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_ptrace": { 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_ptrace", 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_interval": { 2024-04-26T21:26:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_interval", 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:26:49.390 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.7", 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm_log_destination": { 2024-04-26T21:26:49.391 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm_log_destination", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "file", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "file,syslog", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "syslog" 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "cgroups_split": { 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cgroups_split", 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "config_checks_enabled": { 2024-04-26T21:26:49.393 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_checks_enabled", 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: "config_dashboard": { 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_dashboard", 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_alertmanager": { 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_alertmanager", 2024-04-26T21:26:49.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_base": { 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_base", 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.396 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_elasticsearch": { 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:26:49.397 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "elasticsearch container image", 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_grafana": { 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_grafana", 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:26:49.398 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_haproxy": { 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_haproxy", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.399 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "HAproxy container image", 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_agent": { 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:26:49.400 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_collector": { 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.401 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_query": { 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger query container image", 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.402 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_keepalived": { 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_keepalived", 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Keepalived container image", 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_loki": { 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_loki", 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Loki container image", 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_node_exporter": { 2024-04-26T21:26:49.404 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_node_exporter", 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_nvmeof": { 2024-04-26T21:26:49.405 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_nvmeof", 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Nvme-of container image", 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.406 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_prometheus": { 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_prometheus", 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_promtail": { 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_promtail", 2024-04-26T21:26:49.407 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Promtail container image", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_samba": { 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_samba", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.408 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_snmp_gateway": { 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.409 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "container_init": { 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_init", 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.410 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "daemon_cache_timeout": { 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.411 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "900", 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.412 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "default_registry": { 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_registry", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.413 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "device_cache_timeout": { 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_cache_timeout", 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1800", 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "device_enhanced_scan": { 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_enhanced_scan", 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "facts_cache_timeout": { 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "facts_cache_timeout", 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "grafana_dashboards_path": { 2024-04-26T21:26:49.416 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.417 INFO:teuthology.orchestra.run.smithi129.stdout: "host_check_interval": { 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "host_check_interval", 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout: "hw_monitoring": { 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hw_monitoring", 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "inventory_list_all": { 2024-04-26T21:26:49.419 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "inventory_list_all", 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.420 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.421 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "log_refresh_metadata": { 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_refresh_metadata", 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.422 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.423 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.424 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.425 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:26:49.426 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "*", 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.427 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "max_count_per_host": { 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_count_per_host", 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.428 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd_draining_count": { 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_osd_draining_count", 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "migration_current": { 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "migration_current", 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "internal - do not modify", 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "root", 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm-package", 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "root" 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "oob_default_addr": { 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "oob_default_addr", 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "169.254.1.1", 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "prometheus_alerts_path": { 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.433 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_insecure": { 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_insecure", 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_password": { 2024-04-26T21:26:49.434 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_password", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_url": { 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_url", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.435 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_username": { 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_username", 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.436 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "secure_monitoring_stack": { 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.437 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "service_discovery_port": { 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "service_discovery_port", 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.438 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8765", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "ssh_config_file": { 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssh_config_file", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.439 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "use_agent": { 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_agent", 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.440 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "use_repo_digest": { 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_repo_digest", 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:26:49.441 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_failed_host_check": { 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.442 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_daemons": { 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.443 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_hosts": { 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.444 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crash", 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.445 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.446 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.447 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "retain_interval": { 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retain_interval", 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "31536000", 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_recent_interval": { 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_recent_interval", 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1209600", 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dashboard", 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.452 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.453 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.454 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.455 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.456 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.458 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.459 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.460 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.461 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.462 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:26:49.463 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.464 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-04-26T21:26:49.465 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.466 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_URL": { 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.467 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.469 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.470 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.471 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.472 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "[]", 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.473 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "{}", 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.474 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.475 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.476 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.477 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.478 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.479 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.480 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.482 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.483 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:26:49.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:26:49.485 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:26:49.486 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8", 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:26:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "45", 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.489 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.490 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.491 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.492 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.493 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.494 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.495 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "cross_origin_url": { 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cross_origin_url", 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "crt_file": { 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crt_file", 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.497 INFO:teuthology.orchestra.run.smithi129.stdout: "debug": { 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "debug", 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "jwt_token_ttl": { 2024-04-26T21:26:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "jwt_token_ttl", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "28800", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.499 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.500 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.502 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.503 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.504 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "motd": { 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "motd", 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "The message of the day", 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.506 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.507 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8080", 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.508 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl_server_port": { 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl_server_port", 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8443", 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.510 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "redirect", 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect" 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.511 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.512 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "url_prefix": { 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url_prefix", 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.513 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "devicehealth", 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_monitoring": { 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_monitoring", 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.514 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "monitor device health metrics", 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.515 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.518 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.519 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mark_out_threshold": { 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mark_out_threshold", 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2419200", 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.520 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": { 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_name", 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "device_health_metrics", 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.521 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "retention_period": { 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retention_period", 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15552000", 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.522 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_frequency": { 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_frequency", 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.523 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "self_heal": { 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "self_heal", 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.524 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.525 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_threshold": { 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_threshold", 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7257600", 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.526 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "diskprediction_local", 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.527 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.528 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.530 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.531 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "predict_interval": { 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predict_interval", 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.532 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "predictor_model": { 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predictor_model", 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "prophetstor", 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.533 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.534 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "influx", 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": false, 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "batch_size": { 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "batch_size", 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5000", 2024-04-26T21:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "database": { 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "database", 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "ceph", 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.536 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": { 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hostname", 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.537 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "30", 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "5", 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.538 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.539 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.540 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.541 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.543 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "password": { 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "password", 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.544 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "port": { 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "port", 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8086", 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server port", 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "false", 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "threads": { 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threads", 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "32", 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.547 INFO:teuthology.orchestra.run.smithi129.stdout: "username": { 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "username", 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "verify_ssl": { 2024-04-26T21:26:49.548 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "verify_ssl", 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "true", 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.549 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "insights", 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.550 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.551 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.552 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.553 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "iostat", 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.555 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.556 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.557 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.558 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.559 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "k8sevents", 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.560 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_event_retention_days": { 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7", 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.561 INFO:teuthology.orchestra.run.smithi129.stdout: "config_check_secs": { 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_check_secs", 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "10", 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.563 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.564 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.565 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.566 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "localpool", 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.567 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "failure_domain": { 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "failure_domain", 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "host", 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.569 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.570 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.571 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.572 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.573 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": { 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_size", 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:26:49.574 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "num_rep": { 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "num_rep", 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3", 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.575 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": { 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_num", 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "128", 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "prefix": { 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prefix", 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "subtree": { 2024-04-26T21:26:49.577 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "subtree", 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "rack", 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mds_autoscaler", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.579 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.580 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.581 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.582 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirroring", 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.584 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.585 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.586 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.587 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.588 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "nfs", 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.589 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.590 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.592 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.593 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.594 INFO:teuthology.orchestra.run.smithi129.stdout: "fail_fs": { 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "fail_fs", 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.595 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.597 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.598 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.599 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator": { 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-04-26T21:26:49.600 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm", 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rook", 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "test_orchestrator" 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Orchestrator backend", 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.601 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_perf_query", 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.602 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.603 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.605 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.606 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_support", 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.607 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.608 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.609 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.610 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.611 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_autoscaler", 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.612 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.613 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.614 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.615 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.616 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.617 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "threshold": { 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threshold", 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.618 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1.0", 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "scaling threshold", 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "progress", 2024-04-26T21:26:49.619 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_pg_recovery_event": { 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.620 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.622 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.623 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.624 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.625 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "max_completed_events": { 2024-04-26T21:26:49.626 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_completed_events", 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "50", 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.627 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus", 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "cache": { 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cache", 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "exclude_perf_counters": { 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "exclude_perf_counters", 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.630 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.631 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.632 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.633 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.634 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.635 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools": { 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools", 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "300", 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_interval": { 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_interval", 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15.0", 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.638 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.639 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "9283", 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "stale_cache_strategy": { 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stale_cache_strategy", 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "log", 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.641 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "default", 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "default", 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "error" 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.642 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_support", 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.646 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.647 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.648 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_snap_create": { 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.649 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.650 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "trash_purge_schedule": { 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "trash_purge_schedule", 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.651 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "restful", 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_auth": { 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_auth", 2024-04-26T21:26:49.652 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.653 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.654 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.655 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.657 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.658 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.659 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rgw", 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.662 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.663 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.664 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.665 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rook", 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.668 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.669 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "storage_class": { 2024-04-26T21:26:49.671 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "storage_class", 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "local", 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "selftest", 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.673 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.674 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.675 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.676 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.677 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "roption1": { 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption1", 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.678 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "roption2": { 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption2", 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "xyz", 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.679 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption1": { 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption1", 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.680 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption2": { 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption2", 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.681 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption3": { 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption3", 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.682 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption4": { 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption4", 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.683 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption5": { 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption5", 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.684 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption6": { 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption6", 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.685 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption7": { 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption7", 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "42", 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.686 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "testkey": { 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testkey", 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.687 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "testlkey": { 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testlkey", 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.688 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "testnewline": { 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testnewline", 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snap_schedule", 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_m_granularity": { 2024-04-26T21:26:49.690 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_m_granularity", 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "dump_on_update": { 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dump_on_update", 2024-04-26T21:26:49.691 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.692 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.694 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.695 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.696 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stats", 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.697 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.698 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.699 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.700 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.701 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "status", 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.702 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.703 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.704 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.705 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.706 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telegraf", 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "address": { 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "address", 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.708 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15", 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.709 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.710 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.711 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.712 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.714 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telemetry", 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_basic": { 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_basic", 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.715 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_crash": { 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_crash", 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.716 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_device": { 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_device", 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.717 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_ident": { 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_ident", 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.718 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_perf": { 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_perf", 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.719 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "contact": { 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "contact", 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.720 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "description": { 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "description", 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.721 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "device_url": { 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_url", 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.722 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.723 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "24", 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "8", 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.724 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "last_opt_revision": { 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "last_opt_revision", 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1", 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.725 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard": { 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard", 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.726 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard_description": { 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard_description", 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.727 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.728 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.729 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.730 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.731 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.732 INFO:teuthology.orchestra.run.smithi129.stdout: "organization": { 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "organization", 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "proxy": { 2024-04-26T21:26:49.733 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "proxy", 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "url": { 2024-04-26T21:26:49.734 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url", 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "test_orchestrator", 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.736 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.737 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.738 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.739 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.740 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "volumes", 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.741 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.742 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.743 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.744 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.745 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_clones": { 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_clones", 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4", 2024-04-26T21:26:49.746 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "periodic_async_work": { 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "periodic_async_work", 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.747 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Periodically check for async work", 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_delay": { 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.748 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.749 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix", 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "discovery_interval": { 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "discovery_interval", 2024-04-26T21:26:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "100", 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "identifier": { 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "identifier", 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.752 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.753 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.754 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.755 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.756 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.757 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_host": { 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_host", 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.758 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_port": { 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_port", 2024-04-26T21:26:49.759 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10051", 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_sender": { 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_sender", 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.760 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.761 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "modules": [ 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm", 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "dashboard", 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "iostat", 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "nfs", 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "restful" 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "available_modules": [ 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "alerts", 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.762 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.763 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.764 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.765 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.766 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_destination": { 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_destination", 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.768 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_from_name": { 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_from_name", 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "Ceph", 2024-04-26T21:26:49.769 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email From: name", 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_host": { 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_host", 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.770 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP server", 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_password": { 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_password", 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.771 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Password to authenticate with", 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_port": { 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_port", 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "465", 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.772 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP port", 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_sender": { 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_sender", 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.773 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_ssl": { 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_ssl", 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.774 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_user": { 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_user", 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.775 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "User to authenticate as", 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "balancer", 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "active": { 2024-04-26T21:26:49.776 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "active", 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_time": { 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_time", 2024-04-26T21:26:49.777 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0000", 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_weekday": { 2024-04-26T21:26:49.778 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_weekday", 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_max_iterations": { 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:26:49.779 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "25", 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "250", 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_metrics": { 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_metrics", 2024-04-26T21:26:49.780 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_step": { 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_step", 2024-04-26T21:26:49.781 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.5", 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0.001", 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "0.999", 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "end_time": { 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_time", 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.782 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2359", 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "end_weekday": { 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_weekday", 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.783 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.784 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.785 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.786 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.787 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.788 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "min_score": { 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_score", 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.789 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "upmap", 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.790 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-compat", 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "none", 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "read", 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap", 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap-read" 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Balancer mode", 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_ids": { 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_ids", 2024-04-26T21:26:49.791 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.792 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_deviation": { 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_deviation", 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.793 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_optimizations": { 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.794 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm", 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.795 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_down_multiplier": { 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_down_multiplier", 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_refresh_rate": { 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_refresh_rate", 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "20", 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_starting_port": { 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_starting_port", 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4721", 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_ptrace": { 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_ptrace", 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_interval": { 2024-04-26T21:26:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_interval", 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:26:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.7", 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm_log_destination": { 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm_log_destination", 2024-04-26T21:26:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "file", 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "file,syslog", 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "syslog" 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "cgroups_split": { 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cgroups_split", 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "config_checks_enabled": { 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_checks_enabled", 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "config_dashboard": { 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_dashboard", 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_alertmanager": { 2024-04-26T21:26:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_alertmanager", 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_base": { 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_base", 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_elasticsearch": { 2024-04-26T21:26:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "elasticsearch container image", 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_grafana": { 2024-04-26T21:26:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_grafana", 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_haproxy": { 2024-04-26T21:26:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_haproxy", 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "HAproxy container image", 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_agent": { 2024-04-26T21:26:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_collector": { 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:26:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_query": { 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:26:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger query container image", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_keepalived": { 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_keepalived", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Keepalived container image", 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_loki": { 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_loki", 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Loki container image", 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_node_exporter": { 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_node_exporter", 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_nvmeof": { 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_nvmeof", 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Nvme-of container image", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_prometheus": { 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_prometheus", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_promtail": { 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_promtail", 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:26:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Promtail container image", 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_samba": { 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_samba", 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_snmp_gateway": { 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "container_init": { 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_init", 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:26:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "daemon_cache_timeout": { 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "900", 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "default_registry": { 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_registry", 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io", 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "device_cache_timeout": { 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_cache_timeout", 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1800", 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "device_enhanced_scan": { 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_enhanced_scan", 2024-04-26T21:26:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "facts_cache_timeout": { 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "facts_cache_timeout", 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "grafana_dashboards_path": { 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "host_check_interval": { 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "host_check_interval", 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "hw_monitoring": { 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hw_monitoring", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "inventory_list_all": { 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "inventory_list_all", 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "log_refresh_metadata": { 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_refresh_metadata", 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:26:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "*", 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "max_count_per_host": { 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_count_per_host", 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd_draining_count": { 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_osd_draining_count", 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "migration_current": { 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "migration_current", 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "internal - do not modify", 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "root", 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm-package", 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "root" 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "oob_default_addr": { 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "oob_default_addr", 2024-04-26T21:26:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "169.254.1.1", 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "prometheus_alerts_path": { 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_insecure": { 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_insecure", 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_password": { 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_password", 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_url": { 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_url", 2024-04-26T21:26:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_username": { 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_username", 2024-04-26T21:26:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "secure_monitoring_stack": { 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "service_discovery_port": { 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "service_discovery_port", 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8765", 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "ssh_config_file": { 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssh_config_file", 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "use_agent": { 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_agent", 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "use_repo_digest": { 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_repo_digest", 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_failed_host_check": { 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_daemons": { 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_hosts": { 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crash", 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "retain_interval": { 2024-04-26T21:26:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retain_interval", 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "31536000", 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_recent_interval": { 2024-04-26T21:26:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_recent_interval", 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1209600", 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dashboard", 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:26:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:26:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:26:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:26:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_URL": { 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "[]", 2024-04-26T21:26:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "{}", 2024-04-26T21:26:49.886 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.888 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.889 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.890 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.891 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.892 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.893 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.894 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.895 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.897 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.898 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8", 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "45", 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:26:49.900 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:26:49.901 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:26:49.902 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:26:49.903 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:26:49.904 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:26:49.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:26:49.906 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:26:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.910 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "cross_origin_url": { 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cross_origin_url", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.911 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "crt_file": { 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crt_file", 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.912 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "debug": { 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "debug", 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.913 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "jwt_token_ttl": { 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "jwt_token_ttl", 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "28800", 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.914 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.916 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.917 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.918 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.919 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.920 INFO:teuthology.orchestra.run.smithi129.stdout: "motd": { 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "motd", 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "The message of the day", 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:26:49.921 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.922 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.923 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8080", 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.924 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl_server_port": { 2024-04-26T21:26:49.925 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl_server_port", 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8443", 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-04-26T21:26:49.926 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "redirect", 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect" 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.927 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "url_prefix": { 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url_prefix", 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.929 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "devicehealth", 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_monitoring": { 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_monitoring", 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.930 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "monitor device health metrics", 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.931 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.932 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.933 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.935 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "mark_out_threshold": { 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mark_out_threshold", 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2419200", 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.936 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": { 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_name", 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "device_health_metrics", 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.937 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "retention_period": { 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retention_period", 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15552000", 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.938 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_frequency": { 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_frequency", 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.939 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "self_heal": { 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "self_heal", 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.940 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.941 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_threshold": { 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_threshold", 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7257600", 2024-04-26T21:26:49.942 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "diskprediction_local", 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.943 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.944 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.945 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.946 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.947 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "predict_interval": { 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predict_interval", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.948 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "predictor_model": { 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predictor_model", 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.949 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "prophetstor", 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "influx", 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": false, 2024-04-26T21:26:49.951 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "batch_size": { 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "batch_size", 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5000", 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "database": { 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "database", 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "ceph", 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": { 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hostname", 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.954 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "30", 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "5", 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.956 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.957 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.958 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.959 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "password": { 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "password", 2024-04-26T21:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "port": { 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "port", 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.961 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8086", 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server port", 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.962 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "false", 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "threads": { 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threads", 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.963 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "32", 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "username": { 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "username", 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.964 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "verify_ssl": { 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "verify_ssl", 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.965 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "true", 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "insights", 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.968 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.969 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "iostat", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.972 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.973 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.975 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "k8sevents", 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_event_retention_days": { 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7", 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.977 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "config_check_secs": { 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_check_secs", 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:49.978 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "10", 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.979 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.980 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.981 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.983 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "localpool", 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "failure_domain": { 2024-04-26T21:26:49.984 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "failure_domain", 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "host", 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.985 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.986 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.987 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.988 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:49.989 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": { 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_size", 2024-04-26T21:26:49.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "num_rep": { 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "num_rep", 2024-04-26T21:26:49.991 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3", 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": { 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_num", 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:49.992 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "128", 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "prefix": { 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prefix", 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.993 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "subtree": { 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "subtree", 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.994 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "rack", 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mds_autoscaler", 2024-04-26T21:26:49.995 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.996 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.997 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:49.998 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:49.999 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.000 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirroring", 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.001 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.002 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.003 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.004 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "nfs", 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.006 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.007 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.008 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.009 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "fail_fs": { 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "fail_fs", 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.012 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.013 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.014 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.015 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.016 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.017 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator": { 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.018 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm", 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: "rook", 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: "test_orchestrator" 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Orchestrator backend", 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_perf_query", 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.019 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.020 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.021 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.022 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.023 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_support", 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.025 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.026 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.027 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.028 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.029 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_autoscaler", 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.032 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.033 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.034 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.035 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.036 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "threshold": { 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threshold", 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1.0", 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.037 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "scaling threshold", 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "progress", 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_pg_recovery_event": { 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:26:50.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-04-26T21:26:50.039 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.040 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.041 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.042 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.043 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.044 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "max_completed_events": { 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_completed_events", 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "50", 2024-04-26T21:26:50.045 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.046 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus", 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.047 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "cache": { 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cache", 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.048 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "exclude_perf_counters": { 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "exclude_perf_counters", 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.049 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.050 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.051 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.052 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.053 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.054 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools": { 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools", 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:26:50.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "300", 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_interval": { 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_interval", 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:50.056 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15.0", 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.057 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "9283", 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "stale_cache_strategy": { 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stale_cache_strategy", 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "log", 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.060 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "default", 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "default", 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "error" 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.061 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.062 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_support", 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.063 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.064 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.065 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_snap_create": { 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:26:50.068 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "trash_purge_schedule": { 2024-04-26T21:26:50.069 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "trash_purge_schedule", 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "restful", 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_auth": { 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_auth", 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.071 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.072 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.073 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.074 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.075 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.076 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.077 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rgw", 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.080 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.081 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.082 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.083 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.084 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rook", 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.085 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.086 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.087 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.088 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.089 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "storage_class": { 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "storage_class", 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.090 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "local", 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "selftest", 2024-04-26T21:26:50.091 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.094 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.095 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "roption1": { 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption1", 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "roption2": { 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption2", 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "xyz", 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption1": { 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption1", 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.099 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption2": { 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption2", 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption3": { 2024-04-26T21:26:50.100 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption3", 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption4": { 2024-04-26T21:26:50.101 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption4", 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption5": { 2024-04-26T21:26:50.102 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption5", 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption6": { 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption6", 2024-04-26T21:26:50.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption7": { 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption7", 2024-04-26T21:26:50.104 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "42", 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "testkey": { 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testkey", 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.105 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "testlkey": { 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testlkey", 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.106 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "testnewline": { 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testnewline", 2024-04-26T21:26:50.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.108 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snap_schedule", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_m_granularity": { 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_m_granularity", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.109 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "dump_on_update": { 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dump_on_update", 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.110 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.111 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.112 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.113 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.114 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stats", 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.116 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.117 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.118 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.119 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.120 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "status", 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.121 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.122 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.123 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.124 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.125 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.126 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telegraf", 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "address": { 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "address", 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.127 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15", 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.128 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.129 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.130 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.131 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.132 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telemetry", 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_basic": { 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_basic", 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.134 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_crash": { 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_crash", 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.135 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_device": { 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_device", 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.136 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_ident": { 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_ident", 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.137 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_perf": { 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_perf", 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.138 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "contact": { 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "contact", 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.139 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "description": { 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "description", 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.140 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "device_url": { 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_url", 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.141 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.142 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "24", 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "8", 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.143 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "last_opt_revision": { 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "last_opt_revision", 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1", 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard": { 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard", 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.145 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard_description": { 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard_description", 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.146 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.147 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.148 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.149 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.150 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "organization": { 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "organization", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.151 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "proxy": { 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "proxy", 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.152 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "url": { 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url", 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.153 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "test_orchestrator", 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.154 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.155 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.156 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.157 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.158 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "volumes", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.161 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.162 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.163 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_clones": { 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_clones", 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.164 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4", 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "periodic_async_work": { 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "periodic_async_work", 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.165 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Periodically check for async work", 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_delay": { 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.166 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.167 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix", 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "discovery_interval": { 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "discovery_interval", 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.168 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "100", 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "identifier": { 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "identifier", 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.169 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.170 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.171 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.172 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.173 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.174 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.175 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_host": { 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_host", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_port": { 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_port", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-04-26T21:26:50.176 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10051", 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_sender": { 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_sender", 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-04-26T21:26:50.177 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "services": { 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "dashboard": "https://172.21.15.129:8443/" 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.178 INFO:teuthology.orchestra.run.smithi129.stdout: "always_on_modules": { 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "octopus": [ 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "pacific": [ 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-04-26T21:26:50.179 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "quincy": [ 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-04-26T21:26:50.180 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "reef": [ 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-04-26T21:26:50.181 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "squid": [ 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-04-26T21:26:50.182 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: "last_failure_osd_epoch": 3, 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: "active_clients": [ 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "devicehealth", 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:0", 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1747341822 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "libcephsqlite", 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:0", 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3246983948 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_support", 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:0", 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 991725485 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "volumes", 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:0", 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2169219483 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "volumes", 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:0", 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 193023015 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-26T21:26:50.186 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:26:50.188 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:49 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/409150859' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:26:50.188 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:49 smithi150 ceph-mon[32290]: pgmap v615: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 KiB/s wr, 20 op/s 2024-04-26T21:26:50.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f10b80a3c40 msgr2=0x7f10b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:50.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f10b80a3c40 0x7f10b80a6100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f10cc1028d0 tx=0x7f10c800a5e0 comp rx=0 tx=0).stop 2024-04-26T21:26:50.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 msgr2=0x7f10cc104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:50.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 0x7f10cc104460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f10c40097b0 tx=0x7f10c4003c20 comp rx=0 tx=0).stop 2024-04-26T21:26:50.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 shutdown_connections 2024-04-26T21:26:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f10b80a3c40 0x7f10b80a6100 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10cc102a70 0x7f10cc103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10cc103430 0x7f10cc104460 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 --2- 172.21.15.129:0/409150859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10cc101880 0x7f10cc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 >> 172.21.15.129:0/409150859 conn(0x7f10cc0fce10 msgr2=0x7f10cc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 shutdown_connections 2024-04-26T21:26:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:26:48.895+0000 7f10d3d19640 1 -- 172.21.15.129:0/409150859 wait complete. 2024-04-26T21:26:50.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:48 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/713703844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:26:50.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:48 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/329488938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:26:50.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:48 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/2815045573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:26:50.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:48 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1433943672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:26:50.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:49 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/409150859' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:26:50.190 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:49 smithi171 ceph-mon[33478]: pgmap v615: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 KiB/s wr, 20 op/s 2024-04-26T21:26:50.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:48 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/713703844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:26:50.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:48 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/329488938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:26:50.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:48 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/2815045573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:26:50.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:48 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1433943672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:26:50.196 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T21:26:50.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T21:26:50.197 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:49 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/409150859' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:26:50.197 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:49 smithi129 ceph-mon[29237]: pgmap v615: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 KiB/s wr, 20 op/s 2024-04-26T21:26:50.481 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:26:51.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:51 smithi129 ceph-mon[29237]: pgmap v616: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.8 KiB/s wr, 6 op/s 2024-04-26T21:26:51.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:51 smithi150 ceph-mon[32290]: pgmap v616: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.8 KiB/s wr, 6 op/s 2024-04-26T21:26:51.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:51 smithi171 ceph-mon[33478]: pgmap v616: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.8 KiB/s wr, 6 op/s 2024-04-26T21:26:52.601 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 184, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 180, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 333776, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 109456, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.028239537, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.011307218, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.010231580, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.005261403, 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:26:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 180, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.019230982, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000106838 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 333776, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 109456 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 135, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 160, 2024-04-26T21:26:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 102176, 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 3, 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.010021980, 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.007568688, 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.001150803, 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 133, 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.007284093, 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000054767 2024-04-26T21:26:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 102176 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 0, 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 2, 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 0, 2024-04-26T21:26:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 0, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.000184165, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.000121827, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:26:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:26:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:26:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-04-26T21:26:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 148923, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 29, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:26:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:26:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 181, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 305403, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 181, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 181, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 305403, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:52.613 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:26:53.329 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-26T21:26:53.329 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T21:26:53.329 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T21:26:53.576 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:26:53.868 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:53 smithi129 ceph-mon[29237]: pgmap v617: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-26T21:26:53.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:53 smithi150 ceph-mon[32290]: pgmap v617: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-26T21:26:53.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:53 smithi171 ceph-mon[33478]: pgmap v617: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-26T21:26:55.330 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:26:55.330 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:26:55.330 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 4, 2024-04-26T21:26:55.330 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 4, 2024-04-26T21:26:55.330 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 2944, 2024-04-26T21:26:55.330 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 448, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.001459483, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000033086, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.000457566, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000675793, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000225264 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 185, 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 183, 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 332192, 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 112208, 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-04-26T21:26:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-04-26T21:26:55.333 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.026702485, 2024-04-26T21:26:55.333 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.012123741, 2024-04-26T21:26:55.333 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.010009013, 2024-04-26T21:26:55.333 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.003980841, 2024-04-26T21:26:55.333 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:26:55.333 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 182, 2024-04-26T21:26:55.333 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.010900050, 2024-04-26T21:26:55.334 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000059890 2024-04-26T21:26:55.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.334 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:26:55.334 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:55.334 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:55.334 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:55.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 332192, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 112208 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 139, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 160, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 102880, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 3, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.011810217, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.009224772, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.001197586, 2024-04-26T21:26:55.335 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 137, 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.009368894, 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000068386 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 102880 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.336 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:26:55.337 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:26:55.338 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:26:55.339 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 148985, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 30, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-04-26T21:26:55.340 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 186, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 306053, 2024-04-26T21:26:55.341 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 186, 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 186, 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 306053, 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:55.342 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:26:55.691 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:55 smithi129 ceph-mon[29237]: pgmap v618: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:26:55.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:55 smithi150 ceph-mon[32290]: pgmap v618: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:26:55.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:55 smithi171 ceph-mon[33478]: pgmap v618: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:26:56.046 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-26T21:26:56.046 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T21:26:56.046 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T21:26:56.297 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:26:57.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:57 smithi129 ceph-mon[29237]: pgmap v619: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:57.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:57 smithi150 ceph-mon[32290]: pgmap v619: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:57.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:57 smithi171 ceph-mon[33478]: pgmap v619: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:58.289 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:26:58.289 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:26:58.289 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 186, 2024-04-26T21:26:58.289 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 182, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 334320, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 110736, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.026932952, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.011347939, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.009826885, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.003862902, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 182, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.008207526, 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000045096 2024-04-26T21:26:58.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 334320, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 110736 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 136, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 160, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 102352, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 3, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-04-26T21:26:58.291 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.010511487, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.007974819, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.001137431, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 134, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.005957595, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000044459 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:26:58.292 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 102352 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 0, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 2, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 0, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 0, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.000386524, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.000319434, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:58.293 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.294 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:26:58.295 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:26:58.296 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 149024, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 31, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:26:58.297 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:26:58.298 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 183, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 305663, 2024-04-26T21:26:58.299 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 183, 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 183, 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 305663, 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:26:58.300 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:26:58.996 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-26T21:26:58.996 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T21:26:58.996 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T21:26:59.232 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:26:59.614 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:26:59 smithi129 ceph-mon[29237]: pgmap v620: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:59.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:26:59 smithi150 ceph-mon[32290]: pgmap v620: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:59.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:26:59 smithi171 ceph-mon[33478]: pgmap v620: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:01.011 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 188, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 186, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 333008, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 114128, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.028997522, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.012585079, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.010909942, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.004091686, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 185, 2024-04-26T21:27:01.012 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.009002908, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000048664 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 333008, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 114128 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 5, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 143, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 3104, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 103680, 2024-04-26T21:27:01.013 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 3, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.011447541, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.008392060, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.001483484, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 142, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.007770965, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000054725 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:27:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 103680 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 0, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 2, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 0, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 0, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.000292871, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.000216014, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:01.015 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-04-26T21:27:01.016 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:27:01.017 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:27:01.018 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 148967, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 29, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-04-26T21:27:01.019 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:27:01.020 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 189, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 306443, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 189, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 189, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 306443, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:27:01.021 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-26T21:27:01.022 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-26T21:27:01.689 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:01 smithi129 ceph-mon[29237]: pgmap v621: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:01.716 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-26T21:27:01.716 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T21:27:01.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T21:27:01.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:01 smithi150 ceph-mon[32290]: pgmap v621: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:01.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:01 smithi171 ceph-mon[33478]: pgmap v621: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:02.036 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:27:03.838 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:03 smithi129 ceph-mon[29237]: pgmap v622: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:03 smithi150 ceph-mon[32290]: pgmap v622: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:03.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:03 smithi171 ceph-mon[33478]: pgmap v622: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:04.334 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-26T21:27:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:27:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 380057, 2024-04-26T21:27:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 634494, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 121770744, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 550371408, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 13, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 18.948249858, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 9.791074877, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 73.863117859, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 1.850729743, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 634492, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 56.012254509, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000088278 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 505417, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.095034701, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000188 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 90546, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 90539, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 36918356, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 338183334, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 27, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 6.974870672, 2024-04-26T21:27:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 3.036215631, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 1.235655352, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 2.274021802, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 90538, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 43.018241242, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000475140 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 20, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000001677, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000083 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 361104, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 129728 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 38442, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 37933, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 13881294, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 222659764, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 79, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 8, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 4.611511660, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 2.501106852, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.641890586, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 1.225458030, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 37932, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 16.540890449, 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000436066 2024-04-26T21:27:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 841, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000128958, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000153 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 324128, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 360336 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 30204, 2024-04-26T21:27:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 377.945540747, 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.012513095 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 59059, 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 54.164339134, 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000917122 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 188019, 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 188019, 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-26T21:27:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 188019, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 4473.918465563, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.023795033 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 3277, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 198, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 33, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 3329, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 2226, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 1103, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 148, 2024-04-26T21:27:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 5, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 5, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 2, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 272945, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 263421, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-04-26T21:27:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 91020, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 234, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 91020, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 645, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 10, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 635, 2024-04-26T21:27:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 90375, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 42627, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 130050, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 7, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 128719, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 63289, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 7247, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 123217 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-26T21:27:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 1, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 60994, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 60993, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:27:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 10, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 232, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 317477, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 220155, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 208931, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 108546, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-26T21:27:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 11224, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 408, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 280, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 266, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 142, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 9, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 14, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 515073172, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 726636949, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2336, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 711.203248804, 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.304453445 2024-04-26T21:27:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 24, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 61027, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 61003, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 19, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 1831, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 1812, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 3329, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 130947, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 127618, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 5, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 61014, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 61009, 2024-04-26T21:27:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 178240, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 231800 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 194566, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 188019, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 18, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 60799, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 993.007251736, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.016332624 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 5652, 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1986.680773075, 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.351500490 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.350 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 17608, 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 8.850147393, 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000502620 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000280481, 2024-04-26T21:27:04.351 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000140240 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.352 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.353 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.353 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-26T21:27:04.353 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.353 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.353 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 202, 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 57.013962427, 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.282247338 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-26T21:27:04.354 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8270, 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 122.946128375, 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.014866520 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1486, 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2.564581408, 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001725828 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 60421, 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1217.408788908, 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.020148769 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-26T21:27:04.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 196, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 31.265149798, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.159516070 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.996399127, 2024-04-26T21:27:04.357 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.124549890 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-26T21:27:04.358 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 7, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.018269953, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002609993 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001906504, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000635501 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 33365, 2024-04-26T21:27:04.359 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 53.164826378, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001593431 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 2, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 2, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 0, 2024-04-26T21:27:04.360 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 2, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 40386, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 20193, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 597, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:27:04.361 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:27:04.362 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:27:04.363 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 289398, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 108, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 7632, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 161, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-26T21:27:04.364 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 2607850, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 36710, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 186130, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 797747527, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 37, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 186130, 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-26T21:27:04.365 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 186130, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2133.497098757, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.011462403 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 186130, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 240633 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 186130, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 91953, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 94177, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1457, 2024-04-26T21:27:04.366 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 23714, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 923, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 3252, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 414, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 198, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 62894, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 91030, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 47428, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-26T21:27:04.367 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 7, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-26T21:27:04.368 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 88, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 3, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-26T21:27:04.369 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 7308, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 2008 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 40, 2024-04-26T21:27:04.370 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 32, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 62389, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 60993, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 509041, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 131858064, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 509041, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-26T21:27:04.371 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 509041, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 131858064, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:04.372 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 186130, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 803909236, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 186130, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 803909236, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-26T21:27:04.373 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 186130, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 186130, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 186130, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 186130, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.374 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 60993, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 6161709, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 60993, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 916, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 6161709, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.375 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x56218269e500": { 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 317477, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 722442645, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 317477, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2336, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 722442645, 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-26T21:27:04.376 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:04.377 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:04.377 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:04.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:04.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:04.377 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-26T21:27:05.214 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=198 2024-04-26T21:27:05.215 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T21:27:05.215 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T21:27:05.461 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:27:05.876 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:05 smithi150 ceph-mon[32290]: pgmap v623: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:05.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:05 smithi129 ceph-mon[29237]: pgmap v623: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:05.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:05 smithi171 ceph-mon[33478]: pgmap v623: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:07.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:07 smithi150 ceph-mon[32290]: pgmap v624: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:07.711 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-26T21:27:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:27:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4, 2024-04-26T21:27:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4, 2024-04-26T21:27:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2944, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 448, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001391799, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000035388, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000488869, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000714791, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000238263 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 188, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 186, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 333008, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 114128, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.024571914, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.009578762, 2024-04-26T21:27:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.010388860, 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.003960286, 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 185, 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.007191583, 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000038873 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 333008, 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 114128 2024-04-26T21:27:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 141, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 160, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 103232, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.007250290, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005362183, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000620566, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 139, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.005905756, 2024-04-26T21:27:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000042487 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 103232 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-26T21:27:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:27:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-26T21:27:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148984, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 30, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-26T21:27:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-26T21:27:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 189, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 306443, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 189, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 189, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 306443, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:07.721 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-26T21:27:07.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:07 smithi129 ceph-mon[29237]: pgmap v624: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:07.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:07 smithi171 ceph-mon[33478]: pgmap v624: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:08.487 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-04-26T21:27:08.487 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T21:27:08.488 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T21:27:08.731 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:27:09.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:09 smithi150 ceph-mon[32290]: pgmap v625: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:09.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:09 smithi129 ceph-mon[29237]: pgmap v625: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:09.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:09 smithi171 ceph-mon[33478]: pgmap v625: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:10.458 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-26T21:27:10.458 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:27:10.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-04-26T21:27:10.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-04-26T21:27:10.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-04-26T21:27:10.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-04-26T21:27:10.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-04-26T21:27:10.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001143114, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000209562, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 140, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 160, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 103056, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.007376988, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005218820, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000877701, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:10.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 138, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.006167028, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000044688 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 103056 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 190, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 186, 2024-04-26T21:27:10.461 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 335408, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 113296, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.024323175, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.009426086, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.010074373, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.003905626, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 186, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.008629783, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000046396 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:10.462 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 335408, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 113296 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:27:10.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:27:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148967, 2024-04-26T21:27:10.465 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 29, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:27:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 187, 2024-04-26T21:27:10.467 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 306183, 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 187, 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 187, 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 306183, 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:10.468 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-26T21:27:11.181 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.h 2024-04-26T21:27:11.181 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T21:27:11.181 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T21:27:11.422 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:27:11.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:11 smithi150 ceph-mon[32290]: pgmap v626: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:11.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:11 smithi129 ceph-mon[29237]: pgmap v626: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:11.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:11 smithi171 ceph-mon[33478]: pgmap v626: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:13.134 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-26T21:27:13.134 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:27:13.134 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 191, 2024-04-26T21:27:13.134 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 189, 2024-04-26T21:27:13.134 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 333824, 2024-04-26T21:27:13.134 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 116048, 2024-04-26T21:27:13.134 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-04-26T21:27:13.134 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.025274254, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.009391715, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.010669730, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.003910897, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 188, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.007678991, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000040845 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:13.135 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 333824, 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 116048 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 5, 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 146, 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3104, 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 104208, 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-04-26T21:27:13.136 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.007921412, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005359226, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001273163, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 145, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.012039223, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000083029 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 104208 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.000394155, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000333529, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-26T21:27:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:27:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-26T21:27:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148967, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 29, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-26T21:27:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-26T21:27:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 192, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 306833, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 192, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 192, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 306833, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-26T21:27:13.144 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-26T21:27:13.821 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:13 smithi150 ceph-mon[32290]: pgmap v627: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:13.848 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-26T21:27:13.849 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T21:27:13.849 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T21:27:13.876 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:13 smithi171 ceph-mon[33478]: pgmap v627: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:13.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:13 smithi129 ceph-mon[29237]: pgmap v627: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:14.100 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:27:15.712 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:15 smithi171 ceph-mon[33478]: pgmap v628: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:15 smithi150 ceph-mon[32290]: pgmap v628: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:15.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:15 smithi129 ceph-mon[29237]: pgmap v628: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:16.027 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-26T21:27:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:27:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 4, 2024-04-26T21:27:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 4, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 2944, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 448, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.001395107, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000034380, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000498891, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000672673, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000224224 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 142, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 160, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 103408, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 3, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 2, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.009212776, 2024-04-26T21:27:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.006155814, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001574060, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 140, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.008128909, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000058063 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 103408 2024-04-26T21:27:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 188, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 186, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 333008, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 114128, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.027885920, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.011129185, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.011037444, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.004357707, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 185, 2024-04-26T21:27:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.010556190, 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000057060 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 333008, 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 114128 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-26T21:27:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:27:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-26T21:27:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148984, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 30, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-26T21:27:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-26T21:27:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 189, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 306443, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 189, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 189, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 306443, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:16.037 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-26T21:27:16.800 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.c 2024-04-26T21:27:16.801 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T21:27:16.801 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T21:27:17.086 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:27:17.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:17 smithi171 ceph-mon[33478]: pgmap v629: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:17.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:17 smithi150 ceph-mon[32290]: pgmap v629: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:17.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:17 smithi129 ceph-mon[29237]: pgmap v629: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 4, 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 4, 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 2944, 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 448, 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.001395479, 2024-04-26T21:27:18.869 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000037134, 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000545570, 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3, 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000725420, 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000241806 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.870 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 190, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 188, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 333552, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 115408, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.027819238, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.011131547, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.010896624, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.004526142, 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:18.871 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 187, 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.011320186, 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000060535 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 333552, 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 115408 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1, 2024-04-26T21:27:18.872 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 143, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 160, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 103584, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 3, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 2, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.007976973, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.005994700, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000627937, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 141, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.007340339, 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000052059 2024-04-26T21:27:18.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 103584 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-26T21:27:18.874 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:27:18.875 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:27:18.876 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148967, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 29, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:27:18.877 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-26T21:27:18.878 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 191, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 306703, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 191, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-26T21:27:18.879 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-26T21:27:18.880 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 191, 2024-04-26T21:27:18.880 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 306703, 2024-04-26T21:27:18.880 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-26T21:27:18.880 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:18.880 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:18.880 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:18.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:18.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:18.880 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-26T21:27:19.648 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-04-26T21:27:19.649 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T21:27:19.649 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T21:27:19.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:19 smithi150 ceph-mon[32290]: pgmap v630: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:19.887 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:27:19.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:19 smithi129 ceph-mon[29237]: pgmap v630: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:19.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:19 smithi171 ceph-mon[33478]: pgmap v630: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:21.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:21 smithi171 ceph-mon[33478]: pgmap v631: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:21.670 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-26T21:27:21.670 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:27:21.670 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1721, 2024-04-26T21:27:21.670 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1716, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 211556742, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 599284, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.286538138, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.055572351, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.164595084, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.052298665, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1716, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.069614625, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000040567 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.671 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 349776, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 115920 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 947, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1088, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 289686759, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 631742, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-04-26T21:27:21.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.224331979, 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.026112823, 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.167229896, 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.023389211, 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1086, 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.037643309, 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000034662 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 330848 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 833, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 835, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 222543768, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 264901, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.166713581, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.022642302, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.116911719, 2024-04-26T21:27:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.021227206, 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 833, 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.026760092, 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000032124 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-26T21:27:21.675 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4891, 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.411730341, 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000084181 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-26T21:27:21.676 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-04-26T21:27:21.677 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 61018, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 17982, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 43036, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 35329, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 2, 2024-04-26T21:27:21.678 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-04-26T21:27:21.679 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 4, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 234, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:27:21.680 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-04-26T21:27:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:27:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 317477, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-04-26T21:27:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 142, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 27, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 515073172, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 726636949, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 726634116, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 317477 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 24, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 61017, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 60993, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 19, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 1831, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 1812, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 61018, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 127285, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 66267, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-04-26T21:27:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 159384, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 215416 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-04-26T21:27:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-04-26T21:27:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-04-26T21:27:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 2, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 2, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 1715, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:27:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:27:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 238662, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 77, 2024-04-26T21:27:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 7632, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 161, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 43034970, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 244456, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-26T21:27:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 3299, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 26, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 4, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 3299, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3299, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 7.803847937, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002365519 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3299, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3300 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 3299, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 3299, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 2097, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1199, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-04-26T21:27:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 2, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-04-26T21:27:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-04-26T21:27:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 88, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 2, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 11, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 11, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-04-26T21:27:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-04-26T21:27:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 3497, 2024-04-26T21:27:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 723459484, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 3497, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3497, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 723459484, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3299, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 722538619, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3299, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 722538619, 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-26T21:27:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-26T21:27:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3299, 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 3299, 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3299, 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 3299, 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-26T21:27:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x561d49c4e6e0": { 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-26T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-04-26T21:27:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-04-26T21:27:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-26T21:27:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:21.714 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-26T21:27:21.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:21 smithi150 ceph-mon[32290]: pgmap v631: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:21.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:21 smithi129 ceph-mon[29237]: pgmap v631: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:22.469 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-26T21:27:22.469 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T21:27:22.469 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid eccca2b2-0410-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T21:27:22.710 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:27:23.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:23 smithi150 ceph-mon[32290]: pgmap v632: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:23.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:23 smithi171 ceph-mon[33478]: pgmap v632: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:23.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:23 smithi129 ceph-mon[29237]: pgmap v632: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:24.555 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-26T21:27:24.555 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:27:24.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 195, 2024-04-26T21:27:24.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 193, 2024-04-26T21:27:24.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 334912, 2024-04-26T21:27:24.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 118608, 2024-04-26T21:27:24.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-26T21:27:24.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-26T21:27:24.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.030272481, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.011493183, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.011941924, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.004920807, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 192, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.010506353, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000054720 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 334912, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 118608 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 5, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 149, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3104, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 104736, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 3, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 2, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.008739477, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.006188097, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001113466, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:24.557 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 148, 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.010946191, 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000073960 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 104736 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:27:24.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.000264436, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000207692, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:24.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:27:24.560 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:27:24.561 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:27:24.562 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148984, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 30, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:27:24.563 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-26T21:27:24.564 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 196, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 307353, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 196, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 196, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 307353, 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-26T21:27:24.565 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-26T21:27:24.566 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-26T21:27:24.566 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-26T21:27:24.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:24.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-26T21:27:24.566 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-26T21:27:25.249 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-26T21:27:25.250 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T21:27:25.261 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T21:27:25.261 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:27:25.261 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:27:25.283 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-26T21:27:25.283 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:27:25.342 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:27:25.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.490+0000 7f3739996640 1 -- 172.21.15.129:0/56885918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734105df0 msgr2=0x7f37341081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.490+0000 7f3739996640 1 --2- 172.21.15.129:0/56885918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734105df0 0x7f37341081e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3724009900 tx=0x7f372402a410 comp rx=0 tx=0).stop 2024-04-26T21:27:25.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.490+0000 7f3739996640 1 -- 172.21.15.129:0/56885918 shutdown_connections 2024-04-26T21:27:25.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.490+0000 7f3739996640 1 --2- 172.21.15.129:0/56885918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3734108720 0x7f373410ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.490+0000 7f3739996640 1 --2- 172.21.15.129:0/56885918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734105df0 0x7f37341081e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.490+0000 7f3739996640 1 --2- 172.21.15.129:0/56885918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37341034c0 0x7f37341058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.490+0000 7f3739996640 1 -- 172.21.15.129:0/56885918 >> 172.21.15.129:0/56885918 conn(0x7f37340fce10 msgr2=0x7f37340ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.490+0000 7f3739996640 1 -- 172.21.15.129:0/56885918 shutdown_connections 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.490+0000 7f3739996640 1 -- 172.21.15.129:0/56885918 wait complete. 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3739996640 1 Processor -- start 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3739996640 1 -- start start 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3739996640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37341034c0 0x7f3734198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3739996640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3734105df0 0x7f3734199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3739996640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734108720 0x7f37341a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3739996640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f373410e4c0 con 0x7f3734108720 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3739996640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f373410e340 con 0x7f3734105df0 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3739996640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f373410e640 con 0x7f37341034c0 2024-04-26T21:27:25.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f37337fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734108720 0x7f37341a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f37337fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734108720 0x7f37341a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46640/0 (socket says 172.21.15.129:46640) 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f37337fe640 1 -- 172.21.15.129:0/394947524 learned_addr learned my addr 172.21.15.129:0/394947524 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f37327fc640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3734105df0 0x7f3734199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3732ffd640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37341034c0 0x7f3734198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f37337fe640 1 -- 172.21.15.129:0/394947524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37341034c0 msgr2=0x7f3734198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f37337fe640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37341034c0 0x7f3734198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f37337fe640 1 -- 172.21.15.129:0/394947524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3734105df0 msgr2=0x7f3734199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f37337fe640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3734105df0 0x7f3734199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f37337fe640 1 -- 172.21.15.129:0/394947524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37341a0830 con 0x7f3734108720 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.491+0000 7f3732ffd640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37341034c0 0x7f3734198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.492+0000 7f37337fe640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734108720 0x7f37341a0190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f372c00eb10 tx=0x7f372c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.492+0000 7f3738994640 1 -- 172.21.15.129:0/394947524 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c004b20 con 0x7f3734108720 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.492+0000 7f3738994640 1 -- 172.21.15.129:0/394947524 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c004cc0 con 0x7f3734108720 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.492+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37341a0b20 con 0x7f3734108720 2024-04-26T21:27:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.492+0000 7f3738994640 1 -- 172.21.15.129:0/394947524 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c010670 con 0x7f3734108720 2024-04-26T21:27:25.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.492+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37341a1070 con 0x7f3734108720 2024-04-26T21:27:25.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.493+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37340fe900 con 0x7f3734108720 2024-04-26T21:27:25.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.494+0000 7f3738994640 1 -- 172.21.15.129:0/394947524 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f372c005000 con 0x7f3734108720 2024-04-26T21:27:25.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.494+0000 7f3738994640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37200a3c40 0x7f37200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:25.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.495+0000 7f3732ffd640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37200a3c40 0x7f37200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:25.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.495+0000 7f3738994640 1 -- 172.21.15.129:0/394947524 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f372c0d0e10 con 0x7f3734108720 2024-04-26T21:27:25.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.495+0000 7f3732ffd640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37200a3c40 0x7f37200a6100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f373419a0f0 tx=0x7f371c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:25.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.498+0000 7f3738994640 1 -- 172.21.15.129:0/394947524 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f372c054060 con 0x7f3734108720 2024-04-26T21:27:25.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.677+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f373402dca0 con 0x7f3734108720 2024-04-26T21:27:25.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.679+0000 7f3738994640 1 -- 172.21.15.129:0/394947524 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7f372c09d680 con 0x7f3734108720 2024-04-26T21:27:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:27:25.680 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:27:25.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37200a3c40 msgr2=0x7f37200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37200a3c40 0x7f37200a6100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f373419a0f0 tx=0x7f371c005f90 comp rx=0 tx=0).stop 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734108720 msgr2=0x7f37341a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734108720 0x7f37341a0190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f372c00eb10 tx=0x7f372c00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 shutdown_connections 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f37200a3c40 0x7f37200a6100 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3734105df0 0x7f3734199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3734108720 0x7f37341a0190 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 --2- 172.21.15.129:0/394947524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37341034c0 0x7f3734198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 >> 172.21.15.129:0/394947524 conn(0x7f37340fce10 msgr2=0x7f3734103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 shutdown_connections 2024-04-26T21:27:25.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.681+0000 7f3739996640 1 -- 172.21.15.129:0/394947524 wait complete. 2024-04-26T21:27:25.692 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:25 smithi129 ceph-mon[29237]: pgmap v633: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:25.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:21.258164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:16:01.621500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:16:02.592849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:27:25.694 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:27:25.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.794+0000 7f5becb46640 1 -- 172.21.15.129:0/1191583348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 msgr2=0x7f5be81032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.794+0000 7f5becb46640 1 --2- 172.21.15.129:0/1191583348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 0x7f5be81032b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009900 tx=0x7f5bd802a410 comp rx=0 tx=0).stop 2024-04-26T21:27:25.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.794+0000 7f5becb46640 1 -- 172.21.15.129:0/1191583348 shutdown_connections 2024-04-26T21:27:25.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.794+0000 7f5becb46640 1 --2- 172.21.15.129:0/1191583348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5be8101ae0 0x7f5be8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.794+0000 7f5becb46640 1 --2- 172.21.15.129:0/1191583348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5be81024a0 0x7f5be8109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.794+0000 7f5becb46640 1 --2- 172.21.15.129:0/1191583348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 0x7f5be81032b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.794+0000 7f5becb46640 1 -- 172.21.15.129:0/1191583348 >> 172.21.15.129:0/1191583348 conn(0x7f5be80fc570 msgr2=0x7f5be80fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:25.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.794+0000 7f5becb46640 1 -- 172.21.15.129:0/1191583348 shutdown_connections 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.794+0000 7f5becb46640 1 -- 172.21.15.129:0/1191583348 wait complete. 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5becb46640 1 Processor -- start 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5becb46640 1 -- start start 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5becb46640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5be8101ae0 0x7f5be8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5becb46640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5be81024a0 0x7f5be8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5becb46640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 0x7f5be819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5becb46640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5be810d3a0 con 0x7f5be8102e90 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5becb46640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5be810d220 con 0x7f5be8101ae0 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5becb46640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5be810d520 con 0x7f5be81024a0 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5be6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 0x7f5be819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5be6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 0x7f5be819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46662/0 (socket says 172.21.15.129:46662) 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5be6d9d640 1 -- 172.21.15.129:0/116274375 learned_addr learned my addr 172.21.15.129:0/116274375 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:25.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5bdffff640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5be81024a0 0x7f5be8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5be659c640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5be8101ae0 0x7f5be8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5be6d9d640 1 -- 172.21.15.129:0/116274375 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5be81024a0 msgr2=0x7f5be8198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5be6d9d640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5be81024a0 0x7f5be8198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5be6d9d640 1 -- 172.21.15.129:0/116274375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5be8101ae0 msgr2=0x7f5be8197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5be6d9d640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5be8101ae0 0x7f5be8197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.795+0000 7f5be6d9d640 1 -- 172.21.15.129:0/116274375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be819f720 con 0x7f5be8102e90 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.796+0000 7f5be6d9d640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 0x7f5be819f0b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009fd0 tx=0x7f5be000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.796+0000 7f5bdf7fe640 1 -- 172.21.15.129:0/116274375 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0017070 con 0x7f5be8102e90 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.796+0000 7f5bdf7fe640 1 -- 172.21.15.129:0/116274375 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be0003e20 con 0x7f5be8102e90 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.796+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be819fa10 con 0x7f5be8102e90 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.796+0000 7f5bdf7fe640 1 -- 172.21.15.129:0/116274375 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0010460 con 0x7f5be8102e90 2024-04-26T21:27:25.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.796+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be819ff10 con 0x7f5be8102e90 2024-04-26T21:27:25.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.797+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be8108d90 con 0x7f5be8102e90 2024-04-26T21:27:25.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.798+0000 7f5bdf7fe640 1 -- 172.21.15.129:0/116274375 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5be0003970 con 0x7f5be8102e90 2024-04-26T21:27:25.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.798+0000 7f5bdf7fe640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5bd40a3c40 0x7f5bd40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:25.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.799+0000 7f5be659c640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5bd40a3c40 0x7f5bd40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:25.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.799+0000 7f5bdf7fe640 1 -- 172.21.15.129:0/116274375 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5be00d02a0 con 0x7f5be8102e90 2024-04-26T21:27:25.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.799+0000 7f5be659c640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5bd40a3c40 0x7f5bd40a6100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009230 tx=0x7f5bd80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:25.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.803+0000 7f5bdf7fe640 1 -- 172.21.15.129:0/116274375 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5be009c4c0 con 0x7f5be8102e90 2024-04-26T21:27:25.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:25 smithi150 ceph-mon[32290]: pgmap v633: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:25.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.979+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5be802dd20 con 0x7f5be8102e90 2024-04-26T21:27:25.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.979+0000 7f5bdf7fe640 1 -- 172.21.15.129:0/116274375 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f5be00a07c0 con 0x7f5be8102e90 2024-04-26T21:27:25.981 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-26T21:27:25.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.981+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5bd40a3c40 msgr2=0x7f5bd40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.981+0000 7f5becb46640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5bd40a3c40 0x7f5bd40a6100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009230 tx=0x7f5bd80314a0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.981+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 msgr2=0x7f5be819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:25.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.981+0000 7f5becb46640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 0x7f5be819f0b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009fd0 tx=0x7f5be000b260 comp rx=0 tx=0).stop 2024-04-26T21:27:25.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.982+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 shutdown_connections 2024-04-26T21:27:25.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.982+0000 7f5becb46640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f5bd40a3c40 0x7f5bd40a6100 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.982+0000 7f5becb46640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5be8102e90 0x7f5be819f0b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.982+0000 7f5becb46640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5be81024a0 0x7f5be8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.982+0000 7f5becb46640 1 --2- 172.21.15.129:0/116274375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5be8101ae0 0x7f5be8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:25.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.982+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 >> 172.21.15.129:0/116274375 conn(0x7f5be80fc570 msgr2=0x7f5be8105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:25.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.982+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 shutdown_connections 2024-04-26T21:27:25.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:25.982+0000 7f5becb46640 1 -- 172.21.15.129:0/116274375 wait complete. 2024-04-26T21:27:25.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:25 smithi171 ceph-mon[33478]: pgmap v633: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:27:25.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:27:26.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.094+0000 7f01f739e640 1 -- 172.21.15.129:0/3815570180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 msgr2=0x7f01f0103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.094+0000 7f01f739e640 1 --2- 172.21.15.129:0/3815570180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 0x7f01f0103eb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009900 tx=0x7f01ec02a410 comp rx=0 tx=0).stop 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 -- 172.21.15.129:0/3815570180 shutdown_connections 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 --2- 172.21.15.129:0/3815570180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01f01043f0 0x7f01f010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 --2- 172.21.15.129:0/3815570180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 0x7f01f0103eb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 --2- 172.21.15.129:0/3815570180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01f0102840 0x7f01f0102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 -- 172.21.15.129:0/3815570180 >> 172.21.15.129:0/3815570180 conn(0x7f01f00fddd0 msgr2=0x7f01f01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 -- 172.21.15.129:0/3815570180 shutdown_connections 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 -- 172.21.15.129:0/3815570180 wait complete. 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 Processor -- start 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 -- start start 2024-04-26T21:27:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01f0102840 0x7f01f0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.095+0000 7f01f739e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 0x7f01f0198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f739e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01f01043f0 0x7f01f01a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f739e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010e3e0 con 0x7f01f0103a30 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f739e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010e260 con 0x7f01f01043f0 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f739e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010e560 con 0x7f01f0102840 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f5b9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 0x7f01f0198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f5b9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 0x7f01f0198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46680/0 (socket says 172.21.15.129:46680) 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f5b9b640 1 -- 172.21.15.129:0/1895328478 learned_addr learned my addr 172.21.15.129:0/1895328478 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f6b9d640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01f01043f0 0x7f01f01a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f639c640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01f0102840 0x7f01f0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f5b9b640 1 -- 172.21.15.129:0/1895328478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01f0102840 msgr2=0x7f01f0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f5b9b640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01f0102840 0x7f01f0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f5b9b640 1 -- 172.21.15.129:0/1895328478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01f01043f0 msgr2=0x7f01f01a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f5b9b640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01f01043f0 0x7f01f01a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f5b9b640 1 -- 172.21.15.129:0/1895328478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f01a06c0 con 0x7f01f0103a30 2024-04-26T21:27:26.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f5b9b640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 0x7f01f0198fe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f01ec031500 tx=0x7f01ec0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:26.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01e37fe640 1 -- 172.21.15.129:0/1895328478 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec03f030 con 0x7f01f0103a30 2024-04-26T21:27:26.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01e37fe640 1 -- 172.21.15.129:0/1895328478 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ec003800 con 0x7f01f0103a30 2024-04-26T21:27:26.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f01a0950 con 0x7f01f0103a30 2024-04-26T21:27:26.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01e37fe640 1 -- 172.21.15.129:0/1895328478 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec035070 con 0x7f01f0103a30 2024-04-26T21:27:26.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.096+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f01a0ec0 con 0x7f01f0103a30 2024-04-26T21:27:26.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.097+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f0109b60 con 0x7f01f0103a30 2024-04-26T21:27:26.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.098+0000 7f01e37fe640 1 -- 172.21.15.129:0/1895328478 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f01ec0039b0 con 0x7f01f0103a30 2024-04-26T21:27:26.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.099+0000 7f01e37fe640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01dc0a3c40 0x7f01dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.099+0000 7f01e37fe640 1 -- 172.21.15.129:0/1895328478 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f01ec0f0fb0 con 0x7f01f0103a30 2024-04-26T21:27:26.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.099+0000 7f01f639c640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01dc0a3c40 0x7f01dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.099+0000 7f01f639c640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01dc0a3c40 0x7f01dc0a6100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f01f0103890 tx=0x7f01e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:26.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.104+0000 7f01e37fe640 1 -- 172.21.15.129:0/1895328478 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f01ec0bd820 con 0x7f01f0103a30 2024-04-26T21:27:26.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.280+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f01f002dd20 con 0x7f01f0103a30 2024-04-26T21:27:26.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.281+0000 7f01e37fe640 1 -- 172.21.15.129:0/1895328478 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f01ec0bd120 con 0x7f01f0103a30 2024-04-26T21:27:26.282 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-26T21:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01dc0a3c40 msgr2=0x7f01dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01dc0a3c40 0x7f01dc0a6100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f01f0103890 tx=0x7f01e4008040 comp rx=0 tx=0).stop 2024-04-26T21:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 msgr2=0x7f01f0198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 0x7f01f0198fe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f01ec031500 tx=0x7f01ec0043c0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 shutdown_connections 2024-04-26T21:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f01dc0a3c40 0x7f01dc0a6100 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01f01043f0 0x7f01f01a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01f0103a30 0x7f01f0198fe0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 --2- 172.21.15.129:0/1895328478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01f0102840 0x7f01f0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 >> 172.21.15.129:0/1895328478 conn(0x7f01f00fddd0 msgr2=0x7f01f0106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:26.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 shutdown_connections 2024-04-26T21:27:26.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.283+0000 7f01f739e640 1 -- 172.21.15.129:0/1895328478 wait complete. 2024-04-26T21:27:26.295 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:27:26.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.395+0000 7efd12171640 1 -- 172.21.15.129:0/3114427196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c103420 msgr2=0x7efd0c109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.395+0000 7efd12171640 1 --2- 172.21.15.129:0/3114427196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c103420 0x7efd0c109cb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efd04009ca0 tx=0x7efd0402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.395+0000 7efd12171640 1 -- 172.21.15.129:0/3114427196 shutdown_connections 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.395+0000 7efd12171640 1 --2- 172.21.15.129:0/3114427196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd0c102a60 0x7efd0c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.395+0000 7efd12171640 1 --2- 172.21.15.129:0/3114427196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c103420 0x7efd0c109cb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.395+0000 7efd12171640 1 --2- 172.21.15.129:0/3114427196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd0c101870 0x7efd0c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.395+0000 7efd12171640 1 -- 172.21.15.129:0/3114427196 >> 172.21.15.129:0/3114427196 conn(0x7efd0c0fcde0 msgr2=0x7efd0c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.395+0000 7efd12171640 1 -- 172.21.15.129:0/3114427196 shutdown_connections 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd12171640 1 -- 172.21.15.129:0/3114427196 wait complete. 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd12171640 1 Processor -- start 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd12171640 1 -- start start 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd12171640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd0c101870 0x7efd0c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd12171640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c102a60 0x7efd0c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd12171640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd0c103420 0x7efd0c1a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd12171640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10e4c0 con 0x7efd0c102a60 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd12171640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10e340 con 0x7efd0c103420 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd12171640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10e640 con 0x7efd0c101870 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.396+0000 7efd0affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c102a60 0x7efd0c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c102a60 0x7efd0c1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46710/0 (socket says 172.21.15.129:46710) 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0affd640 1 -- 172.21.15.129:0/3858746247 learned_addr learned my addr 172.21.15.129:0/3858746247 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0bfff640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd0c103420 0x7efd0c1a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0b7fe640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd0c101870 0x7efd0c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0affd640 1 -- 172.21.15.129:0/3858746247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd0c101870 msgr2=0x7efd0c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0affd640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd0c101870 0x7efd0c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0affd640 1 -- 172.21.15.129:0/3858746247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd0c103420 msgr2=0x7efd0c1a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0affd640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd0c103420 0x7efd0c1a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0affd640 1 -- 172.21.15.129:0/3858746247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c1a07c0 con 0x7efd0c102a60 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd0affd640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c102a60 0x7efd0c1990e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00d950 tx=0x7efcfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:26.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd08ff9640 1 -- 172.21.15.129:0/3858746247 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc017030 con 0x7efd0c102a60 2024-04-26T21:27:26.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd08ff9640 1 -- 172.21.15.129:0/3858746247 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcfc004020 con 0x7efd0c102a60 2024-04-26T21:27:26.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c1a0ab0 con 0x7efd0c102a60 2024-04-26T21:27:26.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd08ff9640 1 -- 172.21.15.129:0/3858746247 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0047a0 con 0x7efd0c102a60 2024-04-26T21:27:26.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.397+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c1a1000 con 0x7efd0c102a60 2024-04-26T21:27:26.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.398+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0c108c00 con 0x7efd0c102a60 2024-04-26T21:27:26.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.399+0000 7efd08ff9640 1 -- 172.21.15.129:0/3858746247 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efcfc009a30 con 0x7efd0c102a60 2024-04-26T21:27:26.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.400+0000 7efd08ff9640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.400+0000 7efd0b7fe640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.400+0000 7efd08ff9640 1 -- 172.21.15.129:0/3858746247 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7efcfc0cff40 con 0x7efd0c102a60 2024-04-26T21:27:26.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.400+0000 7efd0b7fe640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd00005fd0 tx=0x7efd0000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:26.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.404+0000 7efd08ff9640 1 -- 172.21.15.129:0/3858746247 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efcfc09c7b0 con 0x7efd0c102a60 2024-04-26T21:27:26.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.586+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd0c02dcf0 con 0x7efd0c102a60 2024-04-26T21:27:26.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.587+0000 7efd08ff9640 1 -- 172.21.15.129:0/3858746247 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7efcfc09c0b0 con 0x7efd0c102a60 2024-04-26T21:27:26.588 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-26T21:27:26.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 msgr2=0x7efcf40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd00005fd0 tx=0x7efd0000b390 comp rx=0 tx=0).stop 2024-04-26T21:27:26.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c102a60 msgr2=0x7efd0c1990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c102a60 0x7efd0c1990e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00d950 tx=0x7efcfc00de10 comp rx=0 tx=0).stop 2024-04-26T21:27:26.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 shutdown_connections 2024-04-26T21:27:26.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7efcf40a3c40 0x7efcf40a6100 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd0c103420 0x7efd0c1a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd0c102a60 0x7efd0c1990e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 --2- 172.21.15.129:0/3858746247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd0c101870 0x7efd0c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 >> 172.21.15.129:0/3858746247 conn(0x7efd0c0fcde0 msgr2=0x7efd0c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:26.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 shutdown_connections 2024-04-26T21:27:26.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.589+0000 7efd12171640 1 -- 172.21.15.129:0/3858746247 wait complete. 2024-04-26T21:27:26.602 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:27:26.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 -- 172.21.15.129:0/3929795724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548101870 msgr2=0x7f5548101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 --2- 172.21.15.129:0/3929795724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548101870 0x7f5548101c90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5544009900 tx=0x7f554402a410 comp rx=0 tx=0).stop 2024-04-26T21:27:26.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 -- 172.21.15.129:0/3929795724 shutdown_connections 2024-04-26T21:27:26.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 --2- 172.21.15.129:0/3929795724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5548103420 0x7f5548109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 --2- 172.21.15.129:0/3929795724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5548102a60 0x7f5548102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 --2- 172.21.15.129:0/3929795724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548101870 0x7f5548101c90 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 -- 172.21.15.129:0/3929795724 >> 172.21.15.129:0/3929795724 conn(0x7f55480fce20 msgr2=0x7f55480ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 -- 172.21.15.129:0/3929795724 shutdown_connections 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 -- 172.21.15.129:0/3929795724 wait complete. 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.700+0000 7f55507e7640 1 Processor -- start 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f55507e7640 1 -- start start 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f55507e7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5548101870 0x7f5548198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f55507e7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5548102a60 0x7f5548199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f55507e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548103420 0x7f55481a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f55507e7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f554810d370 con 0x7f5548103420 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f55507e7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f554810d1f0 con 0x7f5548102a60 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f55507e7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f554810d4f0 con 0x7f5548101870 2024-04-26T21:27:26.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554ed84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548103420 0x7f55481a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554ed84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548103420 0x7f55481a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46724/0 (socket says 172.21.15.129:46724) 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554ed84640 1 -- 172.21.15.129:0/3993469267 learned_addr learned my addr 172.21.15.129:0/3993469267 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554dd82640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5548102a60 0x7f5548199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554e583640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5548101870 0x7f5548198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554ed84640 1 -- 172.21.15.129:0/3993469267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5548101870 msgr2=0x7f5548198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554ed84640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5548101870 0x7f5548198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554ed84640 1 -- 172.21.15.129:0/3993469267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5548102a60 msgr2=0x7f5548199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554ed84640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5548102a60 0x7f5548199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554ed84640 1 -- 172.21.15.129:0/3993469267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55481a0830 con 0x7f5548103420 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f554ed84640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548103420 0x7f55481a0190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f554000eaa0 tx=0x7f554000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f553f7fe640 1 -- 172.21.15.129:0/3993469267 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000cdb0 con 0x7f5548103420 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55481a0b20 con 0x7f5548103420 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f553f7fe640 1 -- 172.21.15.129:0/3993469267 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5540005020 con 0x7f5548103420 2024-04-26T21:27:26.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f553f7fe640 1 -- 172.21.15.129:0/3993469267 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55400116d0 con 0x7f5548103420 2024-04-26T21:27:26.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.701+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55481a1020 con 0x7f5548103420 2024-04-26T21:27:26.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.702+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55481a11f0 con 0x7f5548103420 2024-04-26T21:27:26.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.704+0000 7f553f7fe640 1 -- 172.21.15.129:0/3993469267 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f55400051d0 con 0x7f5548103420 2024-04-26T21:27:26.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.704+0000 7f553f7fe640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f55340a3c40 0x7f55340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:26.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.704+0000 7f554e583640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f55340a3c40 0x7f55340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:26.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.704+0000 7f553f7fe640 1 -- 172.21.15.129:0/3993469267 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f55400d0b70 con 0x7f5548103420 2024-04-26T21:27:26.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.705+0000 7f554e583640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f55340a3c40 0x7f55340a6100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f554819a0f0 tx=0x7f55440314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:26.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.708+0000 7f553f7fe640 1 -- 172.21.15.129:0/3993469267 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5540016030 con 0x7f5548103420 2024-04-26T21:27:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:26 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/394947524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:27:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:26 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/116274375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:27:26.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:26 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1895328478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:27:26.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.894+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f55481a13c0 con 0x7f5548103420 2024-04-26T21:27:26.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.894+0000 7f553f7fe640 1 -- 172.21.15.129:0/3993469267 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+597 (secure 0 0 0) 0x7f554009cd90 con 0x7f5548103420 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/948723446 2024-04-27T21:08:55.174047+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/3198429930 2024-04-27T21:08:55.174047+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/948723446 2024-04-27T21:08:55.174047+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/905571244 2024-04-27T21:08:55.174047+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/1433037991 2024-04-27T21:08:28.655399+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/1554108727 2024-04-27T21:08:28.655399+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/1433037991 2024-04-27T21:08:28.655399+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/566668733 2024-04-27T21:08:28.655399+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/841395546 2024-04-27T21:08:28.655399+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/1281940584 2024-04-27T21:08:55.174047+0000 2024-04-26T21:27:26.896 INFO:teuthology.orchestra.run.smithi129.stderr:listed 10 entries 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.896+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f55340a3c40 msgr2=0x7f55340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.896+0000 7f55507e7640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f55340a3c40 0x7f55340a6100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f554819a0f0 tx=0x7f55440314a0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.896+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548103420 msgr2=0x7f55481a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.896+0000 7f55507e7640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548103420 0x7f55481a0190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f554000eaa0 tx=0x7f554000ef60 comp rx=0 tx=0).stop 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.897+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 shutdown_connections 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.897+0000 7f55507e7640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f55340a3c40 0x7f55340a6100 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.897+0000 7f55507e7640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5548102a60 0x7f5548199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.897+0000 7f55507e7640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5548103420 0x7f55481a0190 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.897+0000 7f55507e7640 1 --2- 172.21.15.129:0/3993469267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5548101870 0x7f5548198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.897+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 >> 172.21.15.129:0/3993469267 conn(0x7f55480fce20 msgr2=0x7f554806afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:26.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.897+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 shutdown_connections 2024-04-26T21:27:26.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:26.897+0000 7f55507e7640 1 -- 172.21.15.129:0/3993469267 wait complete. 2024-04-26T21:27:26.908 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:26 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/394947524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:27:26.908 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:26 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/116274375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:27:26.908 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:26 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1895328478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:27:26.909 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:27:26.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:27:26.909 DEBUG:teuthology.orchestra.run.smithi129:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:27:26.949 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi129.front.sepia.ceph.com 2024-04-26T21:27:26.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:27:26.950 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:27:26.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:26 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/394947524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:27:26.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:26 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/116274375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:27:26.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:26 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1895328478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:27:27.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:27:27.025 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-26T21:27:27.080 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:27:27.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:27:27.081 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:27:27.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:27:27.144 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-04-26T21:27:27.235 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T21:27:27.245 INFO:tasks.cephadm:Teardown begin 2024-04-26T21:27:27.246 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:27:27.246 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 -- 172.21.15.129:0/2693012972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f01045e0 msgr2=0x7f68f010ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 --2- 172.21.15.129:0/2693012972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f01045e0 0x7f68f010ae70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009930 tx=0x7f68e002a420 comp rx=0 tx=0).stop 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 -- 172.21.15.129:0/2693012972 shutdown_connections 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 --2- 172.21.15.129:0/2693012972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f0103c20 0x7f68f01040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 --2- 172.21.15.129:0/2693012972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f01045e0 0x7f68f010ae70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 --2- 172.21.15.129:0/2693012972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68f0100c50 0x7f68f0101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 -- 172.21.15.129:0/2693012972 >> 172.21.15.129:0/2693012972 conn(0x7f68f00fc5a0 msgr2=0x7f68f00fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 -- 172.21.15.129:0/2693012972 shutdown_connections 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 -- 172.21.15.129:0/2693012972 wait complete. 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 Processor -- start 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 -- start start 2024-04-26T21:27:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.345+0000 7f68f6cda640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f0100c50 0x7f68f0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f6cda640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f0103c20 0x7f68f0199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f6cda640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68f01045e0 0x7f68f01a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f6cda640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f68f010e4d0 con 0x7f68f0100c50 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f6cda640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f68f010e350 con 0x7f68f0103c20 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f6cda640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f68f010e650 con 0x7f68f01045e0 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f4a76640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f0100c50 0x7f68f0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f4a76640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f0100c50 0x7f68f0198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46738/0 (socket says 172.21.15.129:46738) 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f4a76640 1 -- 172.21.15.129:0/202350395 learned_addr learned my addr 172.21.15.129:0/202350395 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f5277640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68f01045e0 0x7f68f01a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68effff640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f0103c20 0x7f68f0199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f4a76640 1 -- 172.21.15.129:0/202350395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68f01045e0 msgr2=0x7f68f01a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f4a76640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68f01045e0 0x7f68f01a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f4a76640 1 -- 172.21.15.129:0/202350395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f0103c20 msgr2=0x7f68f0199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f4a76640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f0103c20 0x7f68f0199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f4a76640 1 -- 172.21.15.129:0/202350395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f01a0730 con 0x7f68f0100c50 2024-04-26T21:27:27.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f4a76640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f0100c50 0x7f68f0198b10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f68e800eaa0 tx=0x7f68e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:27.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68edffb640 1 -- 172.21.15.129:0/202350395 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e8016070 con 0x7f68f0100c50 2024-04-26T21:27:27.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68edffb640 1 -- 172.21.15.129:0/202350395 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68e8003e20 con 0x7f68f0100c50 2024-04-26T21:27:27.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f01a0a20 con 0x7f68f0100c50 2024-04-26T21:27:27.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.346+0000 7f68edffb640 1 -- 172.21.15.129:0/202350395 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e80116d0 con 0x7f68f0100c50 2024-04-26T21:27:27.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.347+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f01a0f20 con 0x7f68f0100c50 2024-04-26T21:27:27.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.347+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f0109d50 con 0x7f68f0100c50 2024-04-26T21:27:27.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.348+0000 7f68edffb640 1 -- 172.21.15.129:0/202350395 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f68e8019070 con 0x7f68f0100c50 2024-04-26T21:27:27.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.349+0000 7f68edffb640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68dc0a3c40 0x7f68dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:27.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.349+0000 7f68effff640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68dc0a3c40 0x7f68dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:27.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.349+0000 7f68edffb640 1 -- 172.21.15.129:0/202350395 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f68e80d0670 con 0x7f68f0100c50 2024-04-26T21:27:27.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.349+0000 7f68effff640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68dc0a3c40 0x7f68dc0a6100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f68d8005e10 tx=0x7f68d8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:27.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.353+0000 7f68edffb640 1 -- 172.21.15.129:0/202350395 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f68e809cee0 con 0x7f68f0100c50 2024-04-26T21:27:27.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.570+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68f002dad0 con 0x7f68f0100c50 2024-04-26T21:27:27.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.570+0000 7f68edffb640 1 -- 172.21.15.129:0/202350395 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7f68e809c7e0 con 0x7f68f0100c50 2024-04-26T21:27:27.572 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:27:27.572 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:27:27.572 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.572+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68dc0a3c40 msgr2=0x7f68dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.572+0000 7f68f6cda640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68dc0a3c40 0x7f68dc0a6100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f68d8005e10 tx=0x7f68d8005da0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f0100c50 msgr2=0x7f68f0198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f0100c50 0x7f68f0198b10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f68e800eaa0 tx=0x7f68e800ef60 comp rx=0 tx=0).stop 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 shutdown_connections 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f68dc0a3c40 0x7f68dc0a6100 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f0103c20 0x7f68f0199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68f01045e0 0x7f68f01a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 --2- 172.21.15.129:0/202350395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f0100c50 0x7f68f0198b10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 >> 172.21.15.129:0/202350395 conn(0x7f68f00fc5a0 msgr2=0x7f68f0106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 shutdown_connections 2024-04-26T21:27:27.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.573+0000 7f68f6cda640 1 -- 172.21.15.129:0/202350395 wait complete. 2024-04-26T21:27:27.585 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:27:27.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.682+0000 7fd03ef32640 1 -- 172.21.15.129:0/403432550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038103b70 msgr2=0x7fd038103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.682+0000 7fd03ef32640 1 --2- 172.21.15.129:0/403432550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038103b70 0x7fd038103ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd024009900 tx=0x7fd02402a410 comp rx=0 tx=0).stop 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.682+0000 7fd03ef32640 1 -- 172.21.15.129:0/403432550 shutdown_connections 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.682+0000 7fd03ef32640 1 --2- 172.21.15.129:0/403432550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd038104530 0x7fd03810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.682+0000 7fd03ef32640 1 --2- 172.21.15.129:0/403432550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038103b70 0x7fd038103ff0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.682+0000 7fd03ef32640 1 --2- 172.21.15.129:0/403432550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd038102980 0x7fd038102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.682+0000 7fd03ef32640 1 -- 172.21.15.129:0/403432550 >> 172.21.15.129:0/403432550 conn(0x7fd0380fdf10 msgr2=0x7fd038100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.682+0000 7fd03ef32640 1 -- 172.21.15.129:0/403432550 shutdown_connections 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.682+0000 7fd03ef32640 1 -- 172.21.15.129:0/403432550 wait complete. 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ef32640 1 Processor -- start 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ef32640 1 -- start start 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ef32640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038102980 0x7fd038198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ef32640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd038103b70 0x7fd038199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ef32640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd038104530 0x7fd0381a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ef32640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810e4b0 con 0x7fd038102980 2024-04-26T21:27:27.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ef32640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810e330 con 0x7fd038104530 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ef32640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810e630 con 0x7fd038103b70 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ccce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038102980 0x7fd038198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ccce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038102980 0x7fd038198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46760/0 (socket says 172.21.15.129:46760) 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ccce640 1 -- 172.21.15.129:0/795987648 learned_addr learned my addr 172.21.15.129:0/795987648 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd037fff640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd038103b70 0x7fd038199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03d4cf640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd038104530 0x7fd0381a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ccce640 1 -- 172.21.15.129:0/795987648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd038103b70 msgr2=0x7fd038199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ccce640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd038103b70 0x7fd038199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ccce640 1 -- 172.21.15.129:0/795987648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd038104530 msgr2=0x7fd0381a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ccce640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd038104530 0x7fd0381a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03ccce640 1 -- 172.21.15.129:0/795987648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0381a0810 con 0x7fd038102980 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.683+0000 7fd03d4cf640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd038104530 0x7fd0381a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.684+0000 7fd03ccce640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038102980 0x7fd038198bf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00d950 tx=0x7fd02c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:27.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.684+0000 7fd035ffb640 1 -- 172.21.15.129:0/795987648 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c017070 con 0x7fd038102980 2024-04-26T21:27:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.684+0000 7fd035ffb640 1 -- 172.21.15.129:0/795987648 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c009c30 con 0x7fd038102980 2024-04-26T21:27:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.684+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0381a0b00 con 0x7fd038102980 2024-04-26T21:27:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.684+0000 7fd035ffb640 1 -- 172.21.15.129:0/795987648 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c00f620 con 0x7fd038102980 2024-04-26T21:27:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.684+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0381a1000 con 0x7fd038102980 2024-04-26T21:27:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.685+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd038109dc0 con 0x7fd038102980 2024-04-26T21:27:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.686+0000 7fd035ffb640 1 -- 172.21.15.129:0/795987648 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd02c01a070 con 0x7fd038102980 2024-04-26T21:27:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.687+0000 7fd035ffb640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0280a3c40 0x7fd0280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.687+0000 7fd037fff640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0280a3c40 0x7fd0280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.687+0000 7fd035ffb640 1 -- 172.21.15.129:0/795987648 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd02c0d00d0 con 0x7fd038102980 2024-04-26T21:27:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.687+0000 7fd037fff640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0280a3c40 0x7fd0280a6100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd03819a100 tx=0x7fd0240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:27.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.691+0000 7fd035ffb640 1 -- 172.21.15.129:0/795987648 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd02c009de0 con 0x7fd038102980 2024-04-26T21:27:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:27 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3858746247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:27:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:27 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3993469267' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:27:27.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:27 smithi171 ceph-mon[33478]: pgmap v634: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:27 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3858746247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:27:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:27 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3993469267' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:27:27.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:27 smithi150 ceph-mon[32290]: pgmap v634: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:27.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.907+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd03802dad0 con 0x7fd038102980 2024-04-26T21:27:27.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.907+0000 7fd035ffb640 1 -- 172.21.15.129:0/795987648 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11825 (secure 0 0 0) 0x7fd02c09c940 con 0x7fd038102980 2024-04-26T21:27:27.909 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:27:27.909 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4106276863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4106276863},{"type":"v1","addr":"172.21.15.129:6835","nonce":4106276863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/277014387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":277014387},{"type":"v1","addr":"172.21.15.129:6837","nonce":277014387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14661,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/556190328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":556190328},{"type":"v1","addr":"172.21.15.129:6839","nonce":556190328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/861528806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":861528806},{"type":"v1","addr":"172.21.15.129:6841","nonce":861528806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1986119815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1986119815},{"type":"v1","addr":"172.21.15.171:6833","nonce":1986119815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1663214837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1663214837},{"type":"v1","addr":"172.21.15.150:6833","nonce":1663214837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3940507923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3940507923},{"type":"v1","addr":"172.21.15.171:6835","nonce":3940507923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2260848843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2260848843},{"type":"v1","addr":"172.21.15.150:6835","nonce":2260848843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2148122468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2148122468},{"type":"v1","addr":"172.21.15.171:6837","nonce":2148122468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1663379365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1663379365},{"type":"v1","addr":"172.21.15.150:6837","nonce":1663379365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:16:03.677644+0000","modified":"2024-04-26T21:16:08.015827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/2157967831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2157967831},{"type":"v1","addr":"172.21.15.171:6839","nonce":2157967831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/3984403456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3984403456},{"type":"v1","addr":"172.21.15.150:6839","nonce":3984403456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-04-26T21:27:27.909 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 19 2024-04-26T21:27:27.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0280a3c40 msgr2=0x7fd0280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0280a3c40 0x7fd0280a6100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd03819a100 tx=0x7fd0240314a0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038102980 msgr2=0x7fd038198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038102980 0x7fd038198bf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00d950 tx=0x7fd02c00de10 comp rx=0 tx=0).stop 2024-04-26T21:27:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 shutdown_connections 2024-04-26T21:27:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fd0280a3c40 0x7fd0280a6100 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd038104530 0x7fd0381a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd038103b70 0x7fd038199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 --2- 172.21.15.129:0/795987648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd038102980 0x7fd038198bf0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 >> 172.21.15.129:0/795987648 conn(0x7fd0380fdf10 msgr2=0x7fd038106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 shutdown_connections 2024-04-26T21:27:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:27.910+0000 7fd03ef32640 1 -- 172.21.15.129:0/795987648 wait complete. 2024-04-26T21:27:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:27 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3858746247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:27:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:27 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3993469267' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:27:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:27 smithi129 ceph-mon[29237]: pgmap v634: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:27.923 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:27:28.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.019+0000 7f5529637640 1 -- 172.21.15.129:0/929687344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524103440 msgr2=0x7f5524109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.019+0000 7f5529637640 1 --2- 172.21.15.129:0/929687344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524103440 0x7f5524109cd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f551c009ca0 tx=0x7f551c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.019+0000 7f5529637640 1 -- 172.21.15.129:0/929687344 shutdown_connections 2024-04-26T21:27:28.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.019+0000 7f5529637640 1 --2- 172.21.15.129:0/929687344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524103440 0x7f5524109cd0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.019+0000 7f5529637640 1 --2- 172.21.15.129:0/929687344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5524102a80 0x7f5524102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.019+0000 7f5529637640 1 --2- 172.21.15.129:0/929687344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5524101890 0x7f5524101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.019+0000 7f5529637640 1 -- 172.21.15.129:0/929687344 >> 172.21.15.129:0/929687344 conn(0x7f55240fce20 msgr2=0x7f55240ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:28.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.019+0000 7f5529637640 1 -- 172.21.15.129:0/929687344 shutdown_connections 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.019+0000 7f5529637640 1 -- 172.21.15.129:0/929687344 wait complete. 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f5529637640 1 Processor -- start 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f5529637640 1 -- start start 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f5529637640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5524101890 0x7f552410c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f5529637640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524102a80 0x7f552410a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f5529637640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5524103440 0x7f552410ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f5529637640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f552410e4e0 con 0x7f5524102a80 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f5529637640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f552410e360 con 0x7f5524103440 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f5529637640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f552410e660 con 0x7f5524101890 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55227fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524102a80 0x7f552410a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55227fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524102a80 0x7f552410a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46776/0 (socket says 172.21.15.129:46776) 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55227fc640 1 -- 172.21.15.129:0/3323685073 learned_addr learned my addr 172.21.15.129:0/3323685073 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55237fe640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5524103440 0x7f552410ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f5522ffd640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5524101890 0x7f552410c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55227fc640 1 -- 172.21.15.129:0/3323685073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5524101890 msgr2=0x7f552410c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55227fc640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5524101890 0x7f552410c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55227fc640 1 -- 172.21.15.129:0/3323685073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5524103440 msgr2=0x7f552410ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55227fc640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5524103440 0x7f552410ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55227fc640 1 -- 172.21.15.129:0/3323685073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552410b620 con 0x7f5524102a80 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.020+0000 7f55237fe640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5524103440 0x7f552410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.021+0000 7f55227fc640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524102a80 0x7f552410a850 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f551800d770 tx=0x7f551800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.021+0000 7f550bfff640 1 -- 172.21.15.129:0/3323685073 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518017070 con 0x7f5524102a80 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.021+0000 7f550bfff640 1 -- 172.21.15.129:0/3323685073 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55180054d0 con 0x7f5524102a80 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.021+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55241a2c40 con 0x7f5524102a80 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.021+0000 7f550bfff640 1 -- 172.21.15.129:0/3323685073 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518004c30 con 0x7f5524102a80 2024-04-26T21:27:28.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.021+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55241a3140 con 0x7f5524102a80 2024-04-26T21:27:28.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.022+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5524108b50 con 0x7f5524102a80 2024-04-26T21:27:28.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.023+0000 7f550bfff640 1 -- 172.21.15.129:0/3323685073 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5518004e10 con 0x7f5524102a80 2024-04-26T21:27:28.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.023+0000 7f550bfff640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f550c0a3c40 0x7f550c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.023+0000 7f5522ffd640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f550c0a3c40 0x7f550c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.023+0000 7f550bfff640 1 -- 172.21.15.129:0/3323685073 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f55180d0530 con 0x7f5524102a80 2024-04-26T21:27:28.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.024+0000 7f5522ffd640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f550c0a3c40 0x7f550c0a6100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5514005fd0 tx=0x7f5514005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:28.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.027+0000 7f550bfff640 1 -- 172.21.15.129:0/3323685073 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f551809cda0 con 0x7f5524102a80 2024-04-26T21:27:28.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.204+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f552402dca0 con 0x7f5524102a80 2024-04-26T21:27:28.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.206+0000 7f550bfff640 1 -- 172.21.15.129:0/3323685073 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22773 (secure 0 0 0) 0x7f551809c6a0 con 0x7f5524102a80 2024-04-26T21:27:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-26T21:27:28.208 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"eccca2b2-0410-11ef-bc93-c7b262605968","created":"2024-04-26T21:07:57.720565+0000","modified":"2024-04-26T21:16:56.355853+0000","last_up_change":"2024-04-26T21:15:13.328373+0000","last_in_change":"2024-04-26T21:14:52.908763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:21.258164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:16:01.621500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:16:02.592849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03a78174-76a0-4fbb-baf7-222c97cf105c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6803","nonce":3232878544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6805","nonce":3232878544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6809","nonce":3232878544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3232878544},{"type":"v1","addr":"172.21.15.129:6807","nonce":3232878544}]},"public_addr":"172.21.15.129:6803/3232878544","cluster_addr":"172.21.15.129:6805/3232878544","heartbeat_back_addr":"172.21.15.129:6809/3232878544","heartbeat_front_addr":"172.21.15.129:6807/3232878544","state":["exists","up"]},{"osd":1,"uuid":"f8567f6e-0353-446c-8272-3c2d98c95515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6801","nonce":735494308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6803","nonce":735494308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6807","nonce":735494308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":735494308},{"type":"v1","addr":"172.21.15.150:6805","nonce":735494308}]},"public_addr":"172.21.15.150:6801/735494308","cluster_addr":"172.21.15.150:6803/735494308","heartbeat_back_addr":"172.21.15.150:6807/735494308","heartbeat_front_addr":"172.21.15.150:6805/735494308","state":["exists","up"]},{"osd":2,"uuid":"984f85b5-efab-46bb-99c0-e94a53cb512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6801","nonce":1157402243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6803","nonce":1157402243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6807","nonce":1157402243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1157402243},{"type":"v1","addr":"172.21.15.171:6805","nonce":1157402243}]},"public_addr":"172.21.15.171:6801/1157402243","cluster_addr":"172.21.15.171:6803/1157402243","heartbeat_back_addr":"172.21.15.171:6807/1157402243","heartbeat_front_addr":"172.21.15.171:6805/1157402243","state":["exists","up"]},{"osd":3,"uuid":"04e52e54-d5ed-417d-957b-e951ff05b920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6811","nonce":347897303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6813","nonce":347897303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6817","nonce":347897303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":347897303},{"type":"v1","addr":"172.21.15.129:6815","nonce":347897303}]},"public_addr":"172.21.15.129:6811/347897303","cluster_addr":"172.21.15.129:6813/347897303","heartbeat_back_addr":"172.21.15.129:6817/347897303","heartbeat_front_addr":"172.21.15.129:6815/347897303","state":["exists","up"]},{"osd":4,"uuid":"fc2cdc94-fc55-4068-b8a0-0119fcb9dd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6809","nonce":155182298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6811","nonce":155182298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6815","nonce":155182298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":155182298},{"type":"v1","addr":"172.21.15.150:6813","nonce":155182298}]},"public_addr":"172.21.15.150:6809/155182298","cluster_addr":"172.21.15.150:6811/155182298","heartbeat_back_addr":"172.21.15.150:6815/155182298","heartbeat_front_addr":"172.21.15.150:6813/155182298","state":["exists","up"]},{"osd":5,"uuid":"09db0a71-d9ab-4959-9fb8-cee38f030d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6809","nonce":1324309916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6811","nonce":1324309916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6815","nonce":1324309916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1324309916},{"type":"v1","addr":"172.21.15.171:6813","nonce":1324309916}]},"public_addr":"172.21.15.171:6809/1324309916","cluster_addr":"172.21.15.171:6811/1324309916","heartbeat_back_addr":"172.21.15.171:6815/1324309916","heartbeat_front_addr":"172.21.15.171:6813/1324309916","state":["exists","up"]},{"osd":6,"uuid":"50c07cc9-ad4e-4283-9a5a-302eb0a948de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6819","nonce":3546434113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6821","nonce":3546434113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6825","nonce":3546434113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3546434113},{"type":"v1","addr":"172.21.15.129:6823","nonce":3546434113}]},"public_addr":"172.21.15.129:6819/3546434113","cluster_addr":"172.21.15.129:6821/3546434113","heartbeat_back_addr":"172.21.15.129:6825/3546434113","heartbeat_front_addr":"172.21.15.129:6823/3546434113","state":["exists","up"]},{"osd":7,"uuid":"189c2c39-0395-43a9-b250-a1e88d350428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6817","nonce":1143400463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6819","nonce":1143400463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6823","nonce":1143400463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1143400463},{"type":"v1","addr":"172.21.15.150:6821","nonce":1143400463}]},"public_addr":"172.21.15.150:6817/1143400463","cluster_addr":"172.21.15.150:6819/1143400463","heartbeat_back_addr":"172.21.15.150:6823/1143400463","heartbeat_front_addr":"172.21.15.150:6821/1143400463","state":["exists","up"]},{"osd":8,"uuid":"4382cd2f-fd00-45ef-9780-73271a167aa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6817","nonce":2306529186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6819","nonce":2306529186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6823","nonce":2306529186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2306529186},{"type":"v1","addr":"172.21.15.171:6821","nonce":2306529186}]},"public_addr":"172.21.15.171:6817/2306529186","cluster_addr":"172.21.15.171:6819/2306529186","heartbeat_back_addr":"172.21.15.171:6823/2306529186","heartbeat_front_addr":"172.21.15.171:6821/2306529186","state":["exists","up"]},{"osd":9,"uuid":"c961ec13-aa37-4a66-b902-f36e159198b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6827","nonce":3561447119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6829","nonce":3561447119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6833","nonce":3561447119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3561447119},{"type":"v1","addr":"172.21.15.129:6831","nonce":3561447119}]},"public_addr":"172.21.15.129:6827/3561447119","cluster_addr":"172.21.15.129:6829/3561447119","heartbeat_back_addr":"172.21.15.129:6833/3561447119","heartbeat_front_addr":"172.21.15.129:6831/3561447119","state":["exists","up"]},{"osd":10,"uuid":"2eeeec10-5f31-4453-99bb-b705a557e632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6825","nonce":1187635399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6827","nonce":1187635399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6831","nonce":1187635399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":1187635399},{"type":"v1","addr":"172.21.15.150:6829","nonce":1187635399}]},"public_addr":"172.21.15.150:6825/1187635399","cluster_addr":"172.21.15.150:6827/1187635399","heartbeat_back_addr":"172.21.15.150:6831/1187635399","heartbeat_front_addr":"172.21.15.150:6829/1187635399","state":["exists","up"]},{"osd":11,"uuid":"ae01372d-419f-4def-9fcd-3d49eb50b0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6825","nonce":2160214157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6827","nonce":2160214157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6831","nonce":2160214157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2160214157},{"type":"v1","addr":"172.21.15.171:6829","nonce":2160214157}]},"public_addr":"172.21.15.171:6825/2160214157","cluster_addr":"172.21.15.171:6827/2160214157","heartbeat_back_addr":"172.21.15.171:6831/2160214157","heartbeat_front_addr":"172.21.15.171:6829/2160214157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:31.113978+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:10:54.857470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:18.456660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.876261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:08.896614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:12:33.364900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:00.196591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:25.794477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:13:51.303371+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:18.386150+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:14:44.792875+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:15:11.843570+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/3198429930":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/948723446":"2024-04-27T21:08:55.174047+0000","172.21.15.129:0/905571244":"2024-04-27T21:08:55.174047+0000","172.21.15.129:6800/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1554108727":"2024-04-27T21:08:28.655399+0000","172.21.15.129:6801/1433037991":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/566668733":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/841395546":"2024-04-27T21:08:28.655399+0000","172.21.15.129:0/1281940584":"2024-04-27T21:08:55.174047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:27:28.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f550c0a3c40 msgr2=0x7f550c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f550c0a3c40 0x7f550c0a6100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5514005fd0 tx=0x7f5514005ce0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524102a80 msgr2=0x7f552410a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524102a80 0x7f552410a850 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f551800d770 tx=0x7f551800dc30 comp rx=0 tx=0).stop 2024-04-26T21:27:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 shutdown_connections 2024-04-26T21:27:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f550c0a3c40 0x7f550c0a6100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5524103440 0x7f552410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5524102a80 0x7f552410a850 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 --2- 172.21.15.129:0/3323685073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5524101890 0x7f552410c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 >> 172.21.15.129:0/3323685073 conn(0x7f55240fce20 msgr2=0x7f5524105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 shutdown_connections 2024-04-26T21:27:28.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.208+0000 7f5529637640 1 -- 172.21.15.129:0/3323685073 wait complete. 2024-04-26T21:27:28.221 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:27:28.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.318+0000 7f8caed51640 1 -- 172.21.15.129:0/3546558653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 msgr2=0x7f8ca8109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.318+0000 7f8caed51640 1 --2- 172.21.15.129:0/3546558653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 0x7f8ca8109cc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009ca0 tx=0x7f8ca002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.318+0000 7f8caed51640 1 -- 172.21.15.129:0/3546558653 shutdown_connections 2024-04-26T21:27:28.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.318+0000 7f8caed51640 1 --2- 172.21.15.129:0/3546558653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 0x7f8ca8109cc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.318+0000 7f8caed51640 1 --2- 172.21.15.129:0/3546558653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca8102a70 0x7f8ca8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.318+0000 7f8caed51640 1 --2- 172.21.15.129:0/3546558653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca8101880 0x7f8ca8101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.318+0000 7f8caed51640 1 -- 172.21.15.129:0/3546558653 >> 172.21.15.129:0/3546558653 conn(0x7f8ca80fcdf0 msgr2=0x7f8ca80ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:28.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.318+0000 7f8caed51640 1 -- 172.21.15.129:0/3546558653 shutdown_connections 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.318+0000 7f8caed51640 1 -- 172.21.15.129:0/3546558653 wait complete. 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8caed51640 1 Processor -- start 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8caed51640 1 -- start start 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8caed51640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca8101880 0x7f8ca8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8caed51640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca8102a70 0x7f8ca8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8caed51640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 0x7f8ca819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8caed51640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca810d3b0 con 0x7f8ca8103430 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8caed51640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca810d230 con 0x7f8ca8102a70 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8caed51640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca810d530 con 0x7f8ca8101880 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8cad2ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 0x7f8ca819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8cad2ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 0x7f8ca819f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46800/0 (socket says 172.21.15.129:46800) 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8cad2ee640 1 -- 172.21.15.129:0/225858455 learned_addr learned my addr 172.21.15.129:0/225858455 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.319+0000 7f8cacaed640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca8101880 0x7f8ca8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8ca7fff640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca8102a70 0x7f8ca8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8cad2ee640 1 -- 172.21.15.129:0/225858455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca8101880 msgr2=0x7f8ca8197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8cad2ee640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca8101880 0x7f8ca8197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8cad2ee640 1 -- 172.21.15.129:0/225858455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca8102a70 msgr2=0x7f8ca8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8cad2ee640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca8102a70 0x7f8ca8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8cad2ee640 1 -- 172.21.15.129:0/225858455 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca819f700 con 0x7f8ca8103430 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8ca7fff640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca8102a70 0x7f8ca8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8cad2ee640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 0x7f8ca819f090 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0006010 tx=0x7f8ca002f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8ca5ffb640 1 -- 172.21.15.129:0/225858455 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0003cf0 con 0x7f8ca8103430 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8ca5ffb640 1 -- 172.21.15.129:0/225858455 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca00044f0 con 0x7f8ca8103430 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8ca5ffb640 1 -- 172.21.15.129:0/225858455 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca002fd50 con 0x7f8ca8103430 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca819f990 con 0x7f8ca8103430 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.320+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca819fee0 con 0x7f8ca8103430 2024-04-26T21:27:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.321+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca8108d10 con 0x7f8ca8103430 2024-04-26T21:27:28.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.322+0000 7f8ca5ffb640 1 -- 172.21.15.129:0/225858455 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8ca0004040 con 0x7f8ca8103430 2024-04-26T21:27:28.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.323+0000 7f8ca5ffb640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8c980a3c40 0x7f8c980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.323+0000 7f8cacaed640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8c980a3c40 0x7f8c980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.323+0000 7f8ca5ffb640 1 -- 172.21.15.129:0/225858455 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f8ca00f1cb0 con 0x7f8ca8103430 2024-04-26T21:27:28.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.323+0000 7f8cacaed640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8c980a3c40 0x7f8c980a6100 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8ca81028d0 tx=0x7f8c9c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:28.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.326+0000 7f8ca5ffb640 1 -- 172.21.15.129:0/225858455 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8ca00be630 con 0x7f8ca8103430 2024-04-26T21:27:28.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.545+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8ca802dc50 con 0x7f8ca8103430 2024-04-26T21:27:28.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.547+0000 7f8ca5ffb640 1 -- 172.21.15.129:0/225858455 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) ==== 161+0+0 (secure 0 0 0) 0x7f8ca00bdf30 con 0x7f8ca8103430 2024-04-26T21:27:28.548 INFO:teuthology.orchestra.run.smithi129.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:27:28.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8c980a3c40 msgr2=0x7f8c980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8c980a3c40 0x7f8c980a6100 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8ca81028d0 tx=0x7f8c9c00b390 comp rx=0 tx=0).stop 2024-04-26T21:27:28.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 msgr2=0x7f8ca819f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 0x7f8ca819f090 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0006010 tx=0x7f8ca002f460 comp rx=0 tx=0).stop 2024-04-26T21:27:28.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 shutdown_connections 2024-04-26T21:27:28.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f8c980a3c40 0x7f8c980a6100 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca8102a70 0x7f8ca8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ca8103430 0x7f8ca819f090 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 --2- 172.21.15.129:0/225858455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca8101880 0x7f8ca8197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 >> 172.21.15.129:0/225858455 conn(0x7f8ca80fcdf0 msgr2=0x7f8ca8105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:28.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 shutdown_connections 2024-04-26T21:27:28.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.550+0000 7f8caed51640 1 -- 172.21.15.129:0/225858455 wait complete. 2024-04-26T21:27:28.563 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:27:28.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 -- 172.21.15.129:0/1473107771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10103410 msgr2=0x7fed10105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 --2- 172.21.15.129:0/1473107771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10103410 0x7fed10105800 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009900 tx=0x7fed0c02a410 comp rx=0 tx=0).stop 2024-04-26T21:27:28.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 -- 172.21.15.129:0/1473107771 shutdown_connections 2024-04-26T21:27:28.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 --2- 172.21.15.129:0/1473107771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed10105d40 0x7fed10108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 --2- 172.21.15.129:0/1473107771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10108670 0x7fed1010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 --2- 172.21.15.129:0/1473107771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10103410 0x7fed10105800 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 -- 172.21.15.129:0/1473107771 >> 172.21.15.129:0/1473107771 conn(0x7fed100fce00 msgr2=0x7fed100ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 -- 172.21.15.129:0/1473107771 shutdown_connections 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 -- 172.21.15.129:0/1473107771 wait complete. 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 Processor -- start 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.663+0000 7fed1807b640 1 -- start start 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed1807b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10103410 0x7fed10102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed1807b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed10105d40 0x7fed101013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed1807b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10108670 0x7fed10101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed1807b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed1010e4e0 con 0x7fed10108670 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed1807b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fed1010e360 con 0x7fed10105d40 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed1807b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fed1010e660 con 0x7fed10103410 2024-04-26T21:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed16618640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10108670 0x7fed10101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed16618640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10108670 0x7fed10101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46818/0 (socket says 172.21.15.129:46818) 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed16618640 1 -- 172.21.15.129:0/1337460724 learned_addr learned my addr 172.21.15.129:0/1337460724 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed15e17640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10103410 0x7fed10102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed15616640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed10105d40 0x7fed101013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed16618640 1 -- 172.21.15.129:0/1337460724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10103410 msgr2=0x7fed10102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed16618640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10103410 0x7fed10102d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed16618640 1 -- 172.21.15.129:0/1337460724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed10105d40 msgr2=0x7fed101013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed16618640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed10105d40 0x7fed101013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed16618640 1 -- 172.21.15.129:0/1337460724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed10101e40 con 0x7fed10108670 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed15e17640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10103410 0x7fed10102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed16618640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10108670 0x7fed10101900 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fed0800eb10 tx=0x7fed0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed06ffd640 1 -- 172.21.15.129:0/1337460724 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08004b20 con 0x7fed10108670 2024-04-26T21:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed06ffd640 1 -- 172.21.15.129:0/1337460724 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed08004cc0 con 0x7fed10108670 2024-04-26T21:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed10102130 con 0x7fed10108670 2024-04-26T21:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed06ffd640 1 -- 172.21.15.129:0/1337460724 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08010670 con 0x7fed10108670 2024-04-26T21:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.664+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed101a76f0 con 0x7fed10108670 2024-04-26T21:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.665+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed100fe8f0 con 0x7fed10108670 2024-04-26T21:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.666+0000 7fed06ffd640 1 -- 172.21.15.129:0/1337460724 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fed08005000 con 0x7fed10108670 2024-04-26T21:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.667+0000 7fed06ffd640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fecfc0a3cc0 0x7fecfc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.667+0000 7fed15e17640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fecfc0a3cc0 0x7fecfc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.667+0000 7fed06ffd640 1 -- 172.21.15.129:0/1337460724 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8334+0+0 (secure 0 0 0) 0x7fed080d0d70 con 0x7fed10108670 2024-04-26T21:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.668+0000 7fed15e17640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fecfc0a3cc0 0x7fecfc0a6180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed0c000c00 tx=0x7fed0c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:28.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.671+0000 7fed06ffd640 1 -- 172.21.15.129:0/1337460724 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fed08054060 con 0x7fed10108670 2024-04-26T21:27:28.883 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b[62955]: ignoring --setuser ceph since I am not root 2024-04-26T21:27:28.883 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b[62955]: ignoring --setgroup ceph since I am not root 2024-04-26T21:27:28.883 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mds[62959]: -- 172.21.15.150:0/1976098328 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a40aea9a40 con 0x55a40ae7d000 2024-04-26T21:27:28.883 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b[62955]: starting mds.b at 2024-04-26T21:27:28.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/202350395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:27:28.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/795987648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:27:28.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/3323685073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:27:28.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/225858455' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:27:28.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mon[32290]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:27:28.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mon[32290]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:27:28.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/225858455' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:27:28.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mon[32290]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:27:28.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:28 smithi150 ceph-mon[32290]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-26T21:27:28.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:28.888+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fed10063530 con 0x7fed10108670 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a[29233]: 2024-04-26T21:27:28.545+0000 7f54c860c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/202350395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/795987648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/3323685073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/225858455' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-mon[29237]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-mon[29237]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/225858455' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-mon[29237]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:27:28.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:28 smithi129 ceph-mon[29237]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-26T21:27:28.991 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i[64475]: ignoring --setuser ceph since I am not root 2024-04-26T21:27:28.991 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i[64475]: ignoring --setgroup ceph since I am not root 2024-04-26T21:27:28.992 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mds[64479]: -- 172.21.15.171:0/2845712822 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1895f3a40 con 0x55f1895c7000 2024-04-26T21:27:28.992 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i[64475]: starting mds.i at 2024-04-26T21:27:28.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/202350395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:27:28.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/795987648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:27:28.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/3323685073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:27:28.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/225858455' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:27:28.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mon[33478]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:27:28.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mon[33478]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:27:28.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/225858455' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:27:28.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mon[33478]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:27:28.993 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:28 smithi171 ceph-mon[33478]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-26T21:27:29.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.566+0000 7fed06ffd640 1 -- 172.21.15.129:0/1337460724 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fed0809d590 con 0x7fed10108670 2024-04-26T21:27:29.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fecfc0a3cc0 msgr2=0x7fecfc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:29.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fecfc0a3cc0 0x7fecfc0a6180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed0c000c00 tx=0x7fed0c0314a0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10108670 msgr2=0x7fed10101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:29.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10108670 0x7fed10101900 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fed0800eb10 tx=0x7fed0800c2d0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 shutdown_connections 2024-04-26T21:27:29.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7fecfc0a3cc0 0x7fecfc0a6180 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed10105d40 0x7fed101013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed10108670 0x7fed10101900 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 --2- 172.21.15.129:0/1337460724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10103410 0x7fed10102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 >> 172.21.15.129:0/1337460724 conn(0x7fed100fce00 msgr2=0x7fed10103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:29.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 shutdown_connections 2024-04-26T21:27:29.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.569+0000 7fed1807b640 1 -- 172.21.15.129:0/1337460724 wait complete. 2024-04-26T21:27:29.582 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:27:29.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.686+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/3431906994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 msgr2=0x7ff2e0109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:29.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.686+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/3431906994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 0x7ff2e0109cb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009be0 tx=0x7ff2d802a6c0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.686+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/3431906994 shutdown_connections 2024-04-26T21:27:29.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.686+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/3431906994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 0x7ff2e0109cb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.686+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/3431906994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2e0102a60 0x7ff2e0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.686+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/3431906994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e0101870 0x7ff2e0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.686+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/3431906994 >> 172.21.15.129:0/3431906994 conn(0x7ff2e00fcde0 msgr2=0x7ff2e00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:29.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.686+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/3431906994 shutdown_connections 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.686+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/3431906994 wait complete. 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2e4cc8640 1 Processor -- start 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2e4cc8640 1 -- start start 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2e4cc8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e0101870 0x7ff2e006a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2e4cc8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2e0102a60 0x7ff2e019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2e4cc8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 0x7ff2e019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2e4cc8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e010d3a0 con 0x7ff2e0103420 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2e4cc8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e010d220 con 0x7ff2e0101870 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2e4cc8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e010d520 con 0x7ff2e0102a60 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2ded9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 0x7ff2e019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2ded9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 0x7ff2e019ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46846/0 (socket says 172.21.15.129:46846) 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2ded9d640 1 -- 172.21.15.129:0/1455869364 learned_addr learned my addr 172.21.15.129:0/1455869364 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:29.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2ddd9b640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2e0102a60 0x7ff2e019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2de59c640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e0101870 0x7ff2e006a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2ded9d640 1 -- 172.21.15.129:0/1455869364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2e0102a60 msgr2=0x7ff2e019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2ded9d640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2e0102a60 0x7ff2e019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2ded9d640 1 -- 172.21.15.129:0/1455869364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e0101870 msgr2=0x7ff2e006a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2ded9d640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e0101870 0x7ff2e006a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.687+0000 7ff2ded9d640 1 -- 172.21.15.129:0/1455869364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e01a0820 con 0x7ff2e0103420 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.688+0000 7ff2ded9d640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 0x7ff2e019ff10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8034040 tx=0x7ff2d8030e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.688+0000 7ff2cf7fe640 1 -- 172.21.15.129:0/1455869364 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8041020 con 0x7ff2e0103420 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.688+0000 7ff2cf7fe640 1 -- 172.21.15.129:0/1455869364 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d80318f0 con 0x7ff2e0103420 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.688+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e01a0a50 con 0x7ff2e0103420 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.688+0000 7ff2cf7fe640 1 -- 172.21.15.129:0/1455869364 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8036040 con 0x7ff2e0103420 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.688+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e01a0f40 con 0x7ff2e0103420 2024-04-26T21:27:29.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.689+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e0108c60 con 0x7ff2e0103420 2024-04-26T21:27:29.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.690+0000 7ff2cf7fe640 1 -- 172.21.15.129:0/1455869364 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff2d8044020 con 0x7ff2e0103420 2024-04-26T21:27:29.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.691+0000 7ff2cf7fe640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2c80a3c40 0x7ff2c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:29.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.691+0000 7ff2cf7fe640 1 -- 172.21.15.129:0/1455869364 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8334+0+0 (secure 0 0 0) 0x7ff2d80f1870 con 0x7ff2e0103420 2024-04-26T21:27:29.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.691+0000 7ff2de59c640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2c80a3c40 0x7ff2c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:29.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.691+0000 7ff2de59c640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2c80a3c40 0x7ff2c80a6100 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4005d00 tx=0x7ff2d4005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:29.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.695+0000 7ff2cf7fe640 1 -- 172.21.15.129:0/1455869364 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2d804b020 con 0x7ff2e0103420 2024-04-26T21:27:29.855 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:29 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1337460724' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:27:29.855 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:29 smithi129 ceph-mon[29237]: pgmap v636: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2024-04-26T21:27:29.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:29.879+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff2e0063560 con 0x7ff2e0103420 2024-04-26T21:27:29.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:29 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1337460724' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:27:29.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:29 smithi150 ceph-mon[32290]: pgmap v636: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2024-04-26T21:27:29.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:29 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1337460724' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:27:29.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:29 smithi171 ceph-mon[33478]: pgmap v636: 97 pgs: 97 active+clean; 207 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2024-04-26T21:27:30.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.576+0000 7ff2cf7fe640 1 -- 172.21.15.129:0/1455869364 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7ff2d8004670 con 0x7ff2e0103420 2024-04-26T21:27:30.577 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:27:30.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2c80a3c40 msgr2=0x7ff2c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:30.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2c80a3c40 0x7ff2c80a6100 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4005d00 tx=0x7ff2d4005c50 comp rx=0 tx=0).stop 2024-04-26T21:27:30.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 msgr2=0x7ff2e019ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:30.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 0x7ff2e019ff10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8034040 tx=0x7ff2d8030e60 comp rx=0 tx=0).stop 2024-04-26T21:27:30.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 shutdown_connections 2024-04-26T21:27:30.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7ff2c80a3c40 0x7ff2c80a6100 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2e0103420 0x7ff2e019ff10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2e0102a60 0x7ff2e019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 --2- 172.21.15.129:0/1455869364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2e0101870 0x7ff2e006a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 >> 172.21.15.129:0/1455869364 conn(0x7ff2e00fcde0 msgr2=0x7ff2e0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:30.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 shutdown_connections 2024-04-26T21:27:30.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.578+0000 7ff2e4cc8640 1 -- 172.21.15.129:0/1455869364 wait complete. 2024-04-26T21:27:30.594 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:27:30.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 -- 172.21.15.129:0/628933135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080108630 msgr2=0x7f708010aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:30.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 --2- 172.21.15.129:0/628933135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080108630 0x7f708010aa80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7078009ca0 tx=0x7f707802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 -- 172.21.15.129:0/628933135 shutdown_connections 2024-04-26T21:27:30.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 --2- 172.21.15.129:0/628933135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080108630 0x7f708010aa80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 --2- 172.21.15.129:0/628933135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7080105d00 0x7f70801080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 --2- 172.21.15.129:0/628933135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70801033d0 0x7f70801057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 -- 172.21.15.129:0/628933135 >> 172.21.15.129:0/628933135 conn(0x7f70800fcdc0 msgr2=0x7f70800ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 -- 172.21.15.129:0/628933135 shutdown_connections 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 -- 172.21.15.129:0/628933135 wait complete. 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.697+0000 7f7086009640 1 Processor -- start 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f7086009640 1 -- start start 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f7086009640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70801033d0 0x7f7080198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f7086009640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080105d00 0x7f70801990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f7086009640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7080108630 0x7f70801a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f7086009640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f708010e4a0 con 0x7f7080105d00 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f7086009640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f708010e320 con 0x7f70801033d0 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f7086009640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f708010e620 con 0x7f7080108630 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080105d00 0x7f70801990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080105d00 0x7f70801990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46862/0 (socket says 172.21.15.129:46862) 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707effd640 1 -- 172.21.15.129:0/1668009609 learned_addr learned my addr 172.21.15.129:0/1668009609 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-26T21:27:30.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707f7fe640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70801033d0 0x7f7080198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707ffff640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7080108630 0x7f70801a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707effd640 1 -- 172.21.15.129:0/1668009609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7080108630 msgr2=0x7f70801a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707effd640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7080108630 0x7f70801a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707effd640 1 -- 172.21.15.129:0/1668009609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70801033d0 msgr2=0x7f7080198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707effd640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70801033d0 0x7f7080198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707effd640 1 -- 172.21.15.129:0/1668009609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70801a0790 con 0x7f7080105d00 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707ffff640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7080108630 0x7f70801a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707effd640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080105d00 0x7f70801990b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f707400e9f0 tx=0x7f707400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.698+0000 7f707cff9640 1 -- 172.21.15.129:0/1668009609 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707400ccc0 con 0x7f7080105d00 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.699+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70801a0a80 con 0x7f7080105d00 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.699+0000 7f707cff9640 1 -- 172.21.15.129:0/1668009609 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70740054d0 con 0x7f7080105d00 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.699+0000 7f707cff9640 1 -- 172.21.15.129:0/1668009609 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074010430 con 0x7f7080105d00 2024-04-26T21:27:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.699+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70801a0f80 con 0x7f7080105d00 2024-04-26T21:27:30.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.700+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70800fe8b0 con 0x7f7080105d00 2024-04-26T21:27:30.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.701+0000 7f707cff9640 1 -- 172.21.15.129:0/1668009609 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f70740046c0 con 0x7f7080105d00 2024-04-26T21:27:30.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.701+0000 7f707cff9640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70680a3c40 0x7f70680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:30.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.701+0000 7f707cff9640 1 -- 172.21.15.129:0/1668009609 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7838+0+0 (secure 0 0 0) 0x7f70740d0720 con 0x7f7080105d00 2024-04-26T21:27:30.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.701+0000 7f707f7fe640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70680a3c40 0x7f70680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:30.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.702+0000 7f707f7fe640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70680a3c40 0x7f70680a6100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f707000bbe0 tx=0x7f707000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.706+0000 7f707cff9640 1 -- 172.21.15.129:0/1668009609 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f707409d1e0 con 0x7f7080105d00 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: Cluster is now healthy 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1337460724' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.171:6838/350840269,v1:172.21.15.171:6839/350840269] up:boot 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: mds.? [v2:172.21.15.150:6838/3185634410,v1:172.21.15.150:6839/3185634410] up:boot 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: fsmap 12 up:standby 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:27:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:30 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1455869364' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:27:30.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:30.903+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f708002dad0 con 0x7f7080105d00 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: Cluster is now healthy 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1337460724' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.171:6838/350840269,v1:172.21.15.171:6839/350840269] up:boot 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: mds.? [v2:172.21.15.150:6838/3185634410,v1:172.21.15.150:6839/3185634410] up:boot 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: fsmap 12 up:standby 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:27:30.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:30 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1455869364' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:27:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:27:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:27:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: Cluster is now healthy 2024-04-26T21:27:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1337460724' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:27:30.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.171:6838/350840269,v1:172.21.15.171:6839/350840269] up:boot 2024-04-26T21:27:30.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: mds.? [v2:172.21.15.150:6838/3185634410,v1:172.21.15.150:6839/3185634410] up:boot 2024-04-26T21:27:30.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: fsmap 12 up:standby 2024-04-26T21:27:30.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:27:30.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: from='mgr.14152 172.21.15.129:0/2293980384' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:27:30.992 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:30 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1455869364' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:27:31.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.581+0000 7f707cff9640 1 -- 172.21.15.129:0/1668009609 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7f707409cae0 con 0x7f7080105d00 2024-04-26T21:27:31.582 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data' removed 2024-04-26T21:27:31.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.584+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70680a3c40 msgr2=0x7f70680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:31.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.584+0000 7f7086009640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70680a3c40 0x7f70680a6100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f707000bbe0 tx=0x7f707000b390 comp rx=0 tx=0).stop 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.584+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080105d00 msgr2=0x7f70801990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.584+0000 7f7086009640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080105d00 0x7f70801990b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f707400e9f0 tx=0x7f707400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.584+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 shutdown_connections 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.584+0000 7f7086009640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:6800/3856000307,v1:172.21.15.129:6801/3856000307] conn(0x7f70680a3c40 0x7f70680a6100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.584+0000 7f7086009640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7080108630 0x7f70801a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.585+0000 7f7086009640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080105d00 0x7f70801990b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.585+0000 7f7086009640 1 --2- 172.21.15.129:0/1668009609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70801033d0 0x7f7080198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.585+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 >> 172.21.15.129:0/1668009609 conn(0x7f70800fcdc0 msgr2=0x7f7080103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.585+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 shutdown_connections 2024-04-26T21:27:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-26T21:27:31.585+0000 7f7086009640 1 -- 172.21.15.129:0/1668009609 wait complete. 2024-04-26T21:27:31.607 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:27:31.635 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:27:31.663 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:27:31.690 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T21:27:31.690 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:27:31.705 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:27:31.720 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:27:31.745 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T21:27:31.745 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T21:27:31.745 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.a 2024-04-26T21:27:31.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:31 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1455869364' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:27:31.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:31 smithi150 ceph-mon[32290]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:27:31.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:31 smithi150 ceph-mon[32290]: from='client.? 172.21.15.129:0/1668009609' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:27:31.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:31 smithi150 ceph-mon[32290]: pgmap v638: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:27:31.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:31 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1455869364' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:27:31.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:31 smithi129 ceph-mon[29237]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:27:31.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:31 smithi129 ceph-mon[29237]: from='client.? 172.21.15.129:0/1668009609' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:27:31.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:31 smithi129 ceph-mon[29237]: pgmap v638: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:27:31.971 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:31 smithi129 systemd[1]: Stopping Ceph mon.a for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:27:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:31 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1455869364' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:27:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:31 smithi171 ceph-mon[33478]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:27:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:31 smithi171 ceph-mon[33478]: from='client.? 172.21.15.129:0/1668009609' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:27:31.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:31 smithi171 ceph-mon[33478]: pgmap v638: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:27:32.445 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a[29233]: 2024-04-26T21:27:32.101+0000 7f54cde17700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:27:32.445 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a[29233]: 2024-04-26T21:27:32.101+0000 7f54cde17700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T21:27:32.445 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:32 smithi129 podman[91299]: 2024-04-26 21:27:32.153498956 +0000 UTC m=+0.175491888 container died 7a96e43a75754ab6173a4b3092c569ac880dbeb52ee14c49f898a5c2d2b287fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T21:27:32.719 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:32 smithi129 podman[91299]: 2024-04-26 21:27:32.544163619 +0000 UTC m=+0.566156541 container cleanup 7a96e43a75754ab6173a4b3092c569ac880dbeb52ee14c49f898a5c2d2b287fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:27:32.720 INFO:journalctl@ceph.mon.a.smithi129.stdout:Apr 26 21:27:32 smithi129 bash[91299]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-a 2024-04-26T21:27:33.119 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T21:27:33.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:27:33.182 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T21:27:33.182 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T21:27:33.183 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.b 2024-04-26T21:27:33.576 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:33 smithi150 systemd[1]: Stopping Ceph mon.b for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:27:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-b[32286]: 2024-04-26T21:27:33.574+0000 7f000e25d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:27:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-b[32286]: 2024-04-26T21:27:33.574+0000 7f000e25d700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T21:27:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:33 smithi150 podman[66541]: 2024-04-26 21:27:33.789429274 +0000 UTC m=+0.351032103 container died d42a09ce00db59dcb8b6a56dcff0387183651afe0f0e6329fd33f0762375ba63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T21:27:34.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:34 smithi150 podman[66541]: 2024-04-26 21:27:34.191980719 +0000 UTC m=+0.753583547 container cleanup d42a09ce00db59dcb8b6a56dcff0387183651afe0f0e6329fd33f0762375ba63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T21:27:34.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:34 smithi150 bash[66541]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-b 2024-04-26T21:27:34.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T21:27:34.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:34 smithi150 podman[66555]: 2024-04-26 21:27:34.627363655 +0000 UTC m=+0.841010543 container remove d42a09ce00db59dcb8b6a56dcff0387183651afe0f0e6329fd33f0762375ba63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T21:27:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:34 smithi150 systemd[1]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.b.service: Deactivated successfully. 2024-04-26T21:27:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:34 smithi150 systemd[1]: Stopped Ceph mon.b for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:27:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 26 21:27:34 smithi150 systemd[1]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.b.service: Consumed 10.189s CPU time. 2024-04-26T21:27:34.942 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:27:34.942 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T21:27:34.942 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T21:27:34.942 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.c 2024-04-26T21:27:35.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:34 smithi171 systemd[1]: Stopping Ceph mon.c for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:27:35.676 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:35 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-c[33458]: 2024-04-26T21:27:35.241+0000 7fb86dabd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:27:35.676 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:35 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-c[33458]: 2024-04-26T21:27:35.241+0000 7fb86dabd700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T21:27:35.676 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:35 smithi171 podman[68454]: 2024-04-26 21:27:35.425890126 +0000 UTC m=+0.286748648 container died e0a14f2dafd7532d960af766102b4974beb1ec4870fc6c857903910c27cbad00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T21:27:35.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:35 smithi171 podman[68454]: 2024-04-26 21:27:35.767215039 +0000 UTC m=+0.628073567 container cleanup e0a14f2dafd7532d960af766102b4974beb1ec4870fc6c857903910c27cbad00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T21:27:35.991 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:35 smithi171 bash[68454]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-c 2024-04-26T21:27:36.242 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T21:27:36.263 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:35 smithi171 podman[68469]: 2024-04-26 21:27:35.993534626 +0000 UTC m=+0.567375038 container remove e0a14f2dafd7532d960af766102b4974beb1ec4870fc6c857903910c27cbad00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T21:27:36.263 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:36 smithi171 systemd[1]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-26T21:27:36.263 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:36 smithi171 systemd[1]: Stopped Ceph mon.c for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:27:36.263 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 26 21:27:36 smithi171 systemd[1]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mon.c.service: Consumed 10.572s CPU time. 2024-04-26T21:27:36.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:27:36.294 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T21:27:36.295 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T21:27:36.295 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.x 2024-04-26T21:27:36.600 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:27:36 smithi129 systemd[1]: Stopping Ceph mgr.x for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:27:36.917 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:27:36 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x[29464]: 2024-04-26T21:27:36.598+0000 7f2807161700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:27:36.917 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:27:36 smithi129 podman[91419]: 2024-04-26 21:27:36.630220682 +0000 UTC m=+0.152347856 container died 74221cb1dd2d5e10d23d3e4eb297ae834f20a26d609de66397fbdf148b443d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T21:27:37.218 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:27:37 smithi129 podman[91419]: 2024-04-26 21:27:37.016328101 +0000 UTC m=+0.538455268 container cleanup 74221cb1dd2d5e10d23d3e4eb297ae834f20a26d609de66397fbdf148b443d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T21:27:37.218 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Apr 26 21:27:37 smithi129 bash[91419]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-x 2024-04-26T21:27:37.431 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T21:27:37.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:27:37.466 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T21:27:37.466 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T21:27:37.466 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.y 2024-04-26T21:27:37.810 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:27:37 smithi150 systemd[1]: Stopping Ceph mgr.y for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:27:38.099 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:27:37 smithi150 podman[66662]: 2024-04-26 21:27:37.810183355 +0000 UTC m=+0.146798071 container died d9611832263da58b04df994b46702832329c9bee70e8ddf4e776848a21b3369e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:27:38.383 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:27:38 smithi150 podman[66662]: 2024-04-26 21:27:38.189928411 +0000 UTC m=+0.526543125 container cleanup d9611832263da58b04df994b46702832329c9bee70e8ddf4e776848a21b3369e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:27:38.383 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 26 21:27:38 smithi150 bash[66662]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-y 2024-04-26T21:27:38.613 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T21:27:38.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:27:38.646 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T21:27:38.647 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T21:27:38.647 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.z 2024-04-26T21:27:38.977 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:27:38 smithi171 systemd[1]: Stopping Ceph mgr.z for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:27:39.239 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:27:38 smithi171 podman[68579]: 2024-04-26 21:27:38.976168242 +0000 UTC m=+0.140848452 container died 22238896e9156177836bb3e1566f925519f5cc715368ad816dcc8a6936d8add5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T21:27:39.491 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:27:39 smithi171 podman[68579]: 2024-04-26 21:27:39.338919441 +0000 UTC m=+0.503599646 container cleanup 22238896e9156177836bb3e1566f925519f5cc715368ad816dcc8a6936d8add5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:27:39.491 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 26 21:27:39 smithi171 bash[68579]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mgr-z 2024-04-26T21:27:39.809 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T21:27:39.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:27:39.843 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T21:27:39.843 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T21:27:39.843 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.0 2024-04-26T21:27:40.146 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:39 smithi129 systemd[1]: Stopping Ceph osd.0 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:27:40.470 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:40 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0[44451]: 2024-04-26T21:27:40.145+0000 7feeb8df4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:27:40.470 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:40 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0[44451]: 2024-04-26T21:27:40.145+0000 7feeb8df4700 -1 osd.0 91 *** Got signal Terminated *** 2024-04-26T21:27:40.470 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:40 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0[44451]: 2024-04-26T21:27:40.145+0000 7feeb8df4700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:27:45.848 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:45 smithi129 podman[91542]: 2024-04-26 21:27:45.581265376 +0000 UTC m=+5.551476379 container died 583b5fa6eb59c3869ef8007433bd86f3653e7c5ef89ab3476185d71afd62a09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T21:27:46.219 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:45 smithi129 podman[91542]: 2024-04-26 21:27:45.938166166 +0000 UTC m=+5.908377168 container cleanup 583b5fa6eb59c3869ef8007433bd86f3653e7c5ef89ab3476185d71afd62a09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:27:46.220 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:45 smithi129 bash[91542]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0 2024-04-26T21:27:46.600 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:46 smithi129 podman[91561]: 2024-04-26 21:27:46.264744945 +0000 UTC m=+0.681353583 container remove 583b5fa6eb59c3869ef8007433bd86f3653e7c5ef89ab3476185d71afd62a09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T21:27:46.877 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:46 smithi129 podman[91652]: 2024-04-26 21:27:46.502814174 +0000 UTC m=+0.017528558 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:27:46.877 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:46 smithi129 podman[91652]: 2024-04-26 21:27:46.615825134 +0000 UTC m=+0.130539510 container create 746f55957afb2411f145b3c3cf232c439af1bd9b8b642e2345e60550ac2e4f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T21:27:46.878 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:46 smithi129 podman[91652]: 2024-04-26 21:27:46.877106178 +0000 UTC m=+0.391820568 container init 746f55957afb2411f145b3c3cf232c439af1bd9b8b642e2345e60550ac2e4f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:27:47.170 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:46 smithi129 podman[91652]: 2024-04-26 21:27:46.88046148 +0000 UTC m=+0.395175856 container start 746f55957afb2411f145b3c3cf232c439af1bd9b8b642e2345e60550ac2e4f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:27:47.170 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:46 smithi129 podman[91652]: 2024-04-26 21:27:46.9936401 +0000 UTC m=+0.508354490 container attach 746f55957afb2411f145b3c3cf232c439af1bd9b8b642e2345e60550ac2e4f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T21:27:47.470 INFO:journalctl@ceph.osd.0.smithi129.stdout:Apr 26 21:27:47 smithi129 podman[91652]: 2024-04-26 21:27:47.170485398 +0000 UTC m=+0.685199772 container died 746f55957afb2411f145b3c3cf232c439af1bd9b8b642e2345e60550ac2e4f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:27:47.884 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T21:27:47.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:27:47.923 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T21:27:47.923 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T21:27:47.923 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.1 2024-04-26T21:27:48.249 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:47 smithi150 systemd[1]: Stopping Ceph osd.1 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:27:48.633 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1[36880]: 2024-04-26T21:27:48.286+0000 7fe3c05f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:27:48.633 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1[36880]: 2024-04-26T21:27:48.286+0000 7fe3c05f9700 -1 osd.1 91 *** Got signal Terminated *** 2024-04-26T21:27:48.633 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1[36880]: 2024-04-26T21:27:48.286+0000 7fe3c05f9700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:27:54.042 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:53 smithi150 podman[66788]: 2024-04-26 21:27:53.731792908 +0000 UTC m=+5.583869699 container died e2ac813354f80df0265f7be1fd3724bfc529cab001ca992e835a8cb577b96667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T21:27:54.382 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:54 smithi150 podman[66788]: 2024-04-26 21:27:54.145733105 +0000 UTC m=+5.997809903 container cleanup e2ac813354f80df0265f7be1fd3724bfc529cab001ca992e835a8cb577b96667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T21:27:54.383 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:54 smithi150 bash[66788]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1 2024-04-26T21:27:54.744 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:54 smithi150 podman[66816]: 2024-04-26 21:27:54.430809106 +0000 UTC m=+0.698767193 container remove e2ac813354f80df0265f7be1fd3724bfc529cab001ca992e835a8cb577b96667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T21:27:55.001 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:54 smithi150 podman[66908]: 2024-04-26 21:27:54.646449763 +0000 UTC m=+0.018041612 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:27:55.002 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:54 smithi150 podman[66908]: 2024-04-26 21:27:54.756763927 +0000 UTC m=+0.128355767 container create 37a1b7971b9e57c666ba09fab4383b2ce2363867e5d6b9f76b5a25aecee545ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:27:55.002 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:55 smithi150 podman[66908]: 2024-04-26 21:27:55.001531046 +0000 UTC m=+0.373122894 container init 37a1b7971b9e57c666ba09fab4383b2ce2363867e5d6b9f76b5a25aecee545ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:27:55.383 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:55 smithi150 podman[66908]: 2024-04-26 21:27:55.005098509 +0000 UTC m=+0.376690356 container start 37a1b7971b9e57c666ba09fab4383b2ce2363867e5d6b9f76b5a25aecee545ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T21:27:55.383 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:55 smithi150 podman[66908]: 2024-04-26 21:27:55.118351773 +0000 UTC m=+0.489943613 container attach 37a1b7971b9e57c666ba09fab4383b2ce2363867e5d6b9f76b5a25aecee545ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T21:27:55.863 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 26 21:27:55 smithi150 podman[66908]: 2024-04-26 21:27:55.499095122 +0000 UTC m=+0.870686964 container died 37a1b7971b9e57c666ba09fab4383b2ce2363867e5d6b9f76b5a25aecee545ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T21:27:56.233 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T21:27:56.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:27:56.276 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T21:27:56.276 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T21:27:56.276 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.2 2024-04-26T21:27:56.602 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:27:56 smithi171 systemd[1]: Stopping Ceph osd.2 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:27:56.991 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:27:56 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2[38635]: 2024-04-26T21:27:56.601+0000 7fc058ba2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:27:56.991 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:27:56 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2[38635]: 2024-04-26T21:27:56.601+0000 7fc058ba2700 -1 osd.2 91 *** Got signal Terminated *** 2024-04-26T21:27:56.991 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:27:56 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2[38635]: 2024-04-26T21:27:56.601+0000 7fc058ba2700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:28:02.362 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:02 smithi171 podman[68720]: 2024-04-26 21:28:02.057266722 +0000 UTC m=+5.584275728 container died bedd3b483131c96c305a4ba241c94a1357285b0b110a6b445a57042e8bb49e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T21:28:02.741 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:02 smithi171 podman[68720]: 2024-04-26 21:28:02.586384435 +0000 UTC m=+6.113393450 container cleanup bedd3b483131c96c305a4ba241c94a1357285b0b110a6b445a57042e8bb49e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T21:28:02.741 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:02 smithi171 bash[68720]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2 2024-04-26T21:28:03.241 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:02 smithi171 podman[68739]: 2024-04-26 21:28:02.829623652 +0000 UTC m=+0.767511724 container remove bedd3b483131c96c305a4ba241c94a1357285b0b110a6b445a57042e8bb49e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T21:28:03.541 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:03 smithi171 podman[68829]: 2024-04-26 21:28:03.144445738 +0000 UTC m=+0.018389671 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:28:03.541 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:03 smithi171 podman[68829]: 2024-04-26 21:28:03.288730822 +0000 UTC m=+0.162674792 container create bd2fae4ec117ca1ca1dd71673cee8c4210909d1ea79765c46d67307985e083cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T21:28:03.541 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:03 smithi171 podman[68829]: 2024-04-26 21:28:03.541350518 +0000 UTC m=+0.415294444 container init bd2fae4ec117ca1ca1dd71673cee8c4210909d1ea79765c46d67307985e083cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T21:28:03.839 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:03 smithi171 podman[68829]: 2024-04-26 21:28:03.544789718 +0000 UTC m=+0.418733645 container start bd2fae4ec117ca1ca1dd71673cee8c4210909d1ea79765c46d67307985e083cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:28:03.839 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:03 smithi171 podman[68829]: 2024-04-26 21:28:03.649615303 +0000 UTC m=+0.523559237 container attach bd2fae4ec117ca1ca1dd71673cee8c4210909d1ea79765c46d67307985e083cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T21:28:04.120 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 26 21:28:03 smithi171 podman[68829]: 2024-04-26 21:28:03.839071048 +0000 UTC m=+0.713014983 container died bd2fae4ec117ca1ca1dd71673cee8c4210909d1ea79765c46d67307985e083cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:28:04.468 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T21:28:04.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:28:04.507 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T21:28:04.508 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T21:28:04.508 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.3 2024-04-26T21:28:04.826 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:04 smithi129 systemd[1]: Stopping Ceph osd.3 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:28:05.220 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:04 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3[50984]: 2024-04-26T21:28:04.824+0000 7fc008782700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:28:05.220 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:04 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3[50984]: 2024-04-26T21:28:04.824+0000 7fc008782700 -1 osd.3 91 *** Got signal Terminated *** 2024-04-26T21:28:05.220 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:04 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3[50984]: 2024-04-26T21:28:04.824+0000 7fc008782700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:28:06.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:06.346+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:07.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:06 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:06.804+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:07.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:07.309+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:07.819 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:07 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:07.519+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:08.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:07 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:07.844+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:08.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:07 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:07.817+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:08.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:08.337+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:08.742 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:08.453+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:08.866 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:08 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:08.536+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:09.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:08 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:08.847+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:09.219 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:08 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:08.865+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:09.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:09.456+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:09.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:09.301+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:09.889 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:09.519+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:10.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:09 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:09.834+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:10.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:09 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:09.888+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:10.478 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:10 smithi129 podman[91761]: 2024-04-26 21:28:10.323807347 +0000 UTC m=+5.626746386 container died b2b4d4937e82f4496eb6635382865f9925095a6d081822223d56f2f0f3ca089f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T21:28:10.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:10.441+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:10.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:10.302+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:10.747 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:10 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:10.477+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:10.747 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:10 smithi129 podman[91761]: 2024-04-26 21:28:10.725096807 +0000 UTC m=+6.028035846 container cleanup b2b4d4937e82f4496eb6635382865f9925095a6d081822223d56f2f0f3ca089f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:28:10.747 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:10 smithi129 bash[91761]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3 2024-04-26T21:28:11.132 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:10 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:10.834+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:11.220 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:11 smithi129 podman[91778]: 2024-04-26 21:28:11.00984787 +0000 UTC m=+0.679826377 container remove b2b4d4937e82f4496eb6635382865f9925095a6d081822223d56f2f0f3ca089f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T21:28:11.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:10 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:10.873+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:11.673 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:11 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:11.445+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:11.673 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:11 smithi129 podman[91877]: 2024-04-26 21:28:11.271503722 +0000 UTC m=+0.021985188 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:28:11.673 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:11 smithi129 podman[91877]: 2024-04-26 21:28:11.402470432 +0000 UTC m=+0.152951882 container create 75b6a2eb9db8b9509e8af01e8f4eea7f7c693516a5fcd9d1b753b70dfc956f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:28:11.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:11.426+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:11.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:11.300+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:11.965 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:11 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:11.823+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:11.966 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:11 smithi129 podman[91877]: 2024-04-26 21:28:11.672005404 +0000 UTC m=+0.422486874 container init 75b6a2eb9db8b9509e8af01e8f4eea7f7c693516a5fcd9d1b753b70dfc956f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T21:28:11.966 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:11 smithi129 podman[91877]: 2024-04-26 21:28:11.675958846 +0000 UTC m=+0.426440297 container start 75b6a2eb9db8b9509e8af01e8f4eea7f7c693516a5fcd9d1b753b70dfc956f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T21:28:11.966 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:11 smithi129 podman[91877]: 2024-04-26 21:28:11.771701176 +0000 UTC m=+0.522182637 container attach 75b6a2eb9db8b9509e8af01e8f4eea7f7c693516a5fcd9d1b753b70dfc956f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T21:28:12.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:11 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:11.872+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:12.220 INFO:journalctl@ceph.osd.3.smithi129.stdout:Apr 26 21:28:11 smithi129 podman[91877]: 2024-04-26 21:28:11.965549788 +0000 UTC m=+0.716031241 container died 75b6a2eb9db8b9509e8af01e8f4eea7f7c693516a5fcd9d1b753b70dfc956f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T21:28:12.609 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:12 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:12.421+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:12.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T21:28:12.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:28:12.663 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T21:28:12.664 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T21:28:12.664 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.4 2024-04-26T21:28:12.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:12.473+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:12.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:12.340+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:12.976 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:12 smithi150 systemd[1]: Stopping Ceph osd.4 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:28:12.976 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:12 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:12.852+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:13.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:12 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:12.873+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:13.382 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:12 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4[43380]: 2024-04-26T21:28:12.975+0000 7f50b8b74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:28:13.383 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:12 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4[43380]: 2024-04-26T21:28:12.975+0000 7f50b8b74700 -1 osd.4 91 *** Got signal Terminated *** 2024-04-26T21:28:13.383 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:12 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4[43380]: 2024-04-26T21:28:12.975+0000 7f50b8b74700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:28:13.719 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:13 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:13.401+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:13.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:13.444+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:13.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:13.387+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:14.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:13 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:13.913+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:14.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:13 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:13.913+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:14.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:13 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:13.900+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:14.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:14 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:14.373+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:14.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:14 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:14.352+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:14.742 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:14 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:14.415+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:14.742 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:14 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:14.415+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:15.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:14 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:14.921+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:15.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:14 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:14.921+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:15.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:14 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:14.947+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:15.442 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:15 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:15.348+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:15.717 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:15 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:15.440+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:15.718 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:15 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:15.440+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:15.719 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:15 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:15.369+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:15.882 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:15 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:15.382+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:16.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:15 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:15.943+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:16.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:15 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:15.943+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:16.361 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:15 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:15.942+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:16.362 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:15 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:15.942+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:16.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:16 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:16.361+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:16.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:16 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:16.341+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:16.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:16 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:16.341+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:16.743 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:16 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:16.395+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:16.743 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:16 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:16.395+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:16.743 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:16 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:16.366+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:17.315 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:17 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4[43380]: 2024-04-26T21:28:17.102+0000 7f50b5168700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.554091+0000 front 2024-04-26T21:27:51.554066+0000 (oldest deadline 2024-04-26T21:28:16.254273+0000) 2024-04-26T21:28:17.315 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:16 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:16.896+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:17.315 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:16 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:16.896+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:17.321 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:16 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:16.976+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:17.321 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:16 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:16.976+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:17.358 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:16 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:16.999+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:17.630 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:17 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:17.357+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:17.630 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:17 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:17.357+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:17.631 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:17 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:17.358+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:17.631 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:17 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:17.358+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:17.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:17 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:17.314+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:17.719 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:17 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:17.319+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:17.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:17 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:17.319+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:18.312 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:18 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:18.028+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:18.327 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:17 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:17.904+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:18.327 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:17 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:17.904+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:18.328 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:17 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:17.978+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:18.328 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:17 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:17.978+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:18.568 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:18 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:18.310+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:18.568 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:18 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:18.310+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:18.569 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:18 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:18.366+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:18.569 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:18 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:18.366+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:18.632 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:18 smithi150 podman[67013]: 2024-04-26 21:28:18.451977795 +0000 UTC m=+5.599922966 container died 3043108fb9812fc9fc176952297e278823f728e240dfbee0cec57d07be94572e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True) 2024-04-26T21:28:18.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:18 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:18.327+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:18.719 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:18 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:18.327+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:18.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:18 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:18.327+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:19.132 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:18 smithi150 podman[67013]: 2024-04-26 21:28:18.85179806 +0000 UTC m=+5.999743228 container cleanup 3043108fb9812fc9fc176952297e278823f728e240dfbee0cec57d07be94572e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True) 2024-04-26T21:28:19.133 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:18 smithi150 bash[67013]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4 2024-04-26T21:28:19.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:18 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:18.929+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:19.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:18 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:18.929+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:19.219 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:18 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:18.950+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:19.219 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:18 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:18.950+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:19.338 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:18 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:18.998+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:19.432 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:19 smithi150 podman[67031]: 2024-04-26 21:28:19.135862199 +0000 UTC m=+0.675568955 container remove 3043108fb9812fc9fc176952297e278823f728e240dfbee0cec57d07be94572e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T21:28:19.432 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:19 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:19.306+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:19.698 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:19 smithi150 podman[67126]: 2024-04-26 21:28:19.335003674 +0000 UTC m=+0.018221525 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:28:19.698 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:19 smithi150 podman[67126]: 2024-04-26 21:28:19.461936375 +0000 UTC m=+0.145154191 container create 5d2bdce03f1f800c1321881bc181f8148153ab8fe5c49dad494e1a2ba43d3c26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T21:28:19.719 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:19 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:19.301+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:19.719 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:19 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:19.301+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:19.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:19 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:19.340+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:19.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:19 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:19.340+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:19.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:19 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:19.337+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:19.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:19 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:19.337+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:19.993 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:19 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:19.911+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:19.993 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:19 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:19.911+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:19.993 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:19 smithi150 podman[67126]: 2024-04-26 21:28:19.698395654 +0000 UTC m=+0.381613480 container init 5d2bdce03f1f800c1321881bc181f8148153ab8fe5c49dad494e1a2ba43d3c26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T21:28:19.994 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:19 smithi150 podman[67126]: 2024-04-26 21:28:19.701753692 +0000 UTC m=+0.384971509 container start 5d2bdce03f1f800c1321881bc181f8148153ab8fe5c49dad494e1a2ba43d3c26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:28:19.994 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:19 smithi150 podman[67126]: 2024-04-26 21:28:19.789611479 +0000 UTC m=+0.472829316 container attach 5d2bdce03f1f800c1321881bc181f8148153ab8fe5c49dad494e1a2ba43d3c26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T21:28:20.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:19 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:19.904+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:20.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:19 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:19.904+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:20.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:19 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:19.960+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:20.293 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 26 21:28:19 smithi150 podman[67126]: 2024-04-26 21:28:19.993577257 +0000 UTC m=+0.676795088 container died 5d2bdce03f1f800c1321881bc181f8148153ab8fe5c49dad494e1a2ba43d3c26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:28:20.594 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:20 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:20.301+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:20.618 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T21:28:20.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:28:20.656 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T21:28:20.656 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T21:28:20.656 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.5 2024-04-26T21:28:20.683 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:20 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:20.372+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:20.683 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:20 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:20.372+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:20.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:20 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:20.361+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:20.684 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:20 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:20.361+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:20.719 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:20 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:20.277+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:20.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:20 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:20.277+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:20.940 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:20 smithi171 systemd[1]: Stopping Ceph osd.5 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:28:21.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:20 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:20.892+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:21.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:20 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:20.892+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:21.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:20 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:20.940+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:21.241 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:20 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:20.973+0000 7f2150d27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:28:21.241 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:20 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:20.973+0000 7f2150d27700 -1 osd.5 91 *** Got signal Terminated *** 2024-04-26T21:28:21.241 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:20 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:20.973+0000 7f2150d27700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:28:21.306 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:20 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:20.912+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:21.307 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:20 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:20.912+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:21.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:21 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:21.306+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:21.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:21 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:21.233+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:21.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:21 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:21.233+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:21.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:21 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:21.405+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:21.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:21 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:21.405+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:21.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:21 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:21.407+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:21.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:21 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:21.407+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:22.213 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:21 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:21.889+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:22.213 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:21 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:21.889+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:22.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:21 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:21.931+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:22.271 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:22 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:22.270+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:22.272 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:21 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:21.922+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:22.272 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:21 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:21.922+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:22.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:22 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:22.212+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:22.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:22 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:22.212+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:22.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:22 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:22.393+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:22.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:22 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:22.393+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:22.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:22 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:22.454+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:22.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:22 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:22.454+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:23.196 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:22 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:22.844+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:23.197 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:22 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:22.844+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:23.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:22 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:22.958+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:23.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:22 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:22.958+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:23.264 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:22 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:22.923+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:23.264 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:22 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:22.923+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:23.469 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:23 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:23.196+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:23.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:23 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:23.196+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:23.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:23 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:23.264+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:23.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:23 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:23.264+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:23.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:23 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:23.431+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:23.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:23 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:23.431+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:23.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:23 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:23.458+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:23.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:23 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:23.458+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:23.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:23 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:23.458+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:24.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:23 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:23.860+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:24.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:23 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:23.860+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:24.260 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:23 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:23.903+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:24.260 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:23 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:23.903+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:24.260 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:23 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:23.903+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:24.457 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:24.007+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:24.457 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:24.007+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:24.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:24 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:24.259+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:24.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:24 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:24.259+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:24.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:24.222+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:24.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:24.222+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:24.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:24.456+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:24.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:24.456+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:24.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:24.456+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:58.931071+0000 front 2024-04-26T21:27:58.931118+0000 (oldest deadline 2024-04-26T21:28:24.231251+0000) 2024-04-26T21:28:24.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:24.505+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:24.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:24.505+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:24.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:24.505+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:25.212 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:24.812+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:25.212 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:24.812+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:25.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:24.974+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:25.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:24 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:24.974+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:25.279 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:24 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:24.935+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:25.279 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:24 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:24.935+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:25.279 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:24 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:24.935+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:25.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:25 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:25.211+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:25.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:25 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:25.211+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:25.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:25 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:25.211+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:25.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:25 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:25.278+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:25.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:25 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:25.278+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:25.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:25 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:25.456+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.826314+0000 front 2024-04-26T21:27:44.826386+0000 (oldest deadline 2024-04-26T21:28:07.626813+0000) 2024-04-26T21:28:25.742 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:25 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:25.456+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.529085+0000 front 2024-04-26T21:27:51.529209+0000 (oldest deadline 2024-04-26T21:28:13.829230+0000) 2024-04-26T21:28:25.742 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:25 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5[45245]: 2024-04-26T21:28:25.456+0000 7f214d31b700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:58.931071+0000 front 2024-04-26T21:27:58.931118+0000 (oldest deadline 2024-04-26T21:28:24.231251+0000) 2024-04-26T21:28:25.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:25 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:25.473+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:25.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:25 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:25.473+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:25.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:25 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:25.473+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:26.219 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:25 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:25.777+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:26.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:25 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:25.777+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:26.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:25 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:25.777+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.283176+0000 front 2024-04-26T21:28:00.283405+0000 (oldest deadline 2024-04-26T21:28:24.983118+0000) 2024-04-26T21:28:26.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:25 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:25.978+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:26.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:25 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:25.978+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:26.247 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:25 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:25.910+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:26.247 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:25 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:25.910+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:26.247 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:25 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:25.910+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:26.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:26 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:26.247+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:26.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:26 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:26.247+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:26.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:26.253+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:26.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:26.253+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:26.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:26.253+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:26.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:26 smithi171 podman[68959]: 2024-04-26 21:28:26.522553688 +0000 UTC m=+5.675409999 container died f4e620d41876de5e76aaefe1a1b0be153d1d08d1a47007d8b48ae561f94a0557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:28:26.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:26 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:26.485+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:26.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:26 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:26.485+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:26.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:26 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:26.485+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:27.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:26.802+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:27.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:26.802+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:27.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:26 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:26.802+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.283176+0000 front 2024-04-26T21:28:00.283405+0000 (oldest deadline 2024-04-26T21:28:24.983118+0000) 2024-04-26T21:28:27.238 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:26 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:26.952+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:27.238 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:26 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:26.952+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:27.238 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:26 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:26.952+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:27.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:26 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:26.997+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:27.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:26 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:26.997+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:27.241 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:26 smithi171 podman[68959]: 2024-04-26 21:28:26.831949938 +0000 UTC m=+5.984806248 container cleanup f4e620d41876de5e76aaefe1a1b0be153d1d08d1a47007d8b48ae561f94a0557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T21:28:27.241 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:26 smithi171 bash[68959]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5 2024-04-26T21:28:27.241 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:27 smithi171 podman[68977]: 2024-04-26 21:28:27.06646305 +0000 UTC m=+0.541994552 container remove f4e620d41876de5e76aaefe1a1b0be153d1d08d1a47007d8b48ae561f94a0557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-26T21:28:27.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:27 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:27.237+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:27.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:27 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:27.237+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:27.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:27.269+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:27.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:27.269+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:27.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:27.269+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:27.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:27 smithi171 podman[69070]: 2024-04-26 21:28:27.405099017 +0000 UTC m=+0.018333870 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:28:27.741 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:27 smithi171 podman[69070]: 2024-04-26 21:28:27.563813818 +0000 UTC m=+0.177048668 container create e5a7905c6240eadbf11e93161e755d76ccb8444f5b14b72a6c13d6ea93390401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T21:28:27.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:27 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:27.484+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:27.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:27 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:27.484+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:27.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:27 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:27.484+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:28.161 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:28 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:28.036+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:28.161 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:28 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:28.036+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:28.162 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:27 smithi171 podman[69070]: 2024-04-26 21:28:27.862225398 +0000 UTC m=+0.475460247 container init e5a7905c6240eadbf11e93161e755d76ccb8444f5b14b72a6c13d6ea93390401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:28:28.162 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:27 smithi171 podman[69070]: 2024-04-26 21:28:27.865694915 +0000 UTC m=+0.478929771 container start e5a7905c6240eadbf11e93161e755d76ccb8444f5b14b72a6c13d6ea93390401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T21:28:28.162 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:27 smithi171 podman[69070]: 2024-04-26 21:28:27.945452941 +0000 UTC m=+0.558687810 container attach e5a7905c6240eadbf11e93161e755d76ccb8444f5b14b72a6c13d6ea93390401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T21:28:28.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:27.797+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:28.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:27.797+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:28.220 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:27 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:27.797+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.283176+0000 front 2024-04-26T21:28:00.283405+0000 (oldest deadline 2024-04-26T21:28:24.983118+0000) 2024-04-26T21:28:28.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:28 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:28.195+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:28.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:28 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:28.195+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:28.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:27 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:27.962+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:28.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:27 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:27.962+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:28.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:27 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:27.962+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:28.432 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 26 21:28:28 smithi171 podman[69095]: 2024-04-26 21:28:28.186323636 +0000 UTC m=+0.018902511 container died e5a7905c6240eadbf11e93161e755d76ccb8444f5b14b72a6c13d6ea93390401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:28:28.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:28 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:28.222+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:28.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:28 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:28.222+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:28.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:28 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:28.222+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:28.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:28 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:28.462+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:28.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:28 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:28.462+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:28.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:28 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:28.462+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:28.771 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T21:28:28.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:28:28.849 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T21:28:28.849 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T21:28:28.850 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.6 2024-04-26T21:28:29.162 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:28 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:28.765+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:29.162 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:28 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:28.765+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:29.162 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:28 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:28.765+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.283176+0000 front 2024-04-26T21:28:00.283405+0000 (oldest deadline 2024-04-26T21:28:24.983118+0000) 2024-04-26T21:28:29.162 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:28 smithi129 systemd[1]: Stopping Ceph osd.6 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:28:29.162 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:29 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:29.161+0000 7f86c6dec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:28:29.163 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:29 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:29.161+0000 7f86c6dec700 -1 osd.6 91 *** Got signal Terminated *** 2024-04-26T21:28:29.163 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:29 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:29.161+0000 7f86c6dec700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:28:29.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:29 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:29.151+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:29.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:29 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:29.151+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:29.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:28 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:28.941+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:29.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:28 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:28.941+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:29.384 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:28 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:28.941+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:29.428 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:29 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:29.069+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:29.428 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:29 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:29.069+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:29.469 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:29 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:29.174+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:29.469 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:29 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:29.174+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:29.469 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:29 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:29.174+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:29.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:29 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:29.428+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:29.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:29 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:29.428+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:29.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:29 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:29.428+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:30.214 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:29 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:29.794+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:30.214 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:29 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:29.794+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:30.214 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:29 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:29.794+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.283176+0000 front 2024-04-26T21:28:00.283405+0000 (oldest deadline 2024-04-26T21:28:24.983118+0000) 2024-04-26T21:28:30.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:30 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:30.129+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:30.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:30 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:30.129+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:30.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:29 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:29.920+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:30.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:29 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:29.920+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:30.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:29 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:29.920+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:30.386 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:30 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:30.107+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:30.386 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:30 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:30.107+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:30.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:30 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:30.213+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:30.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:30 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:30.213+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:30.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:30 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:30.213+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:30.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:30 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:30.386+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:30.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:30 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:30.386+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:30.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:30 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:30.386+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:31.213 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:30 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:30.813+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:31.213 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:30 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:30.813+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:31.213 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:30 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:30.813+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.283176+0000 front 2024-04-26T21:28:00.283405+0000 (oldest deadline 2024-04-26T21:28:24.983118+0000) 2024-04-26T21:28:31.377 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:31 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:31.094+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:31.377 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:31 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:31.094+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:31.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:31 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:31.121+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:31.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:31 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:31.121+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:31.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:30 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:30.970+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:31.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:30 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:30.970+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:31.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:30 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:30.970+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:31.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:31 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:31.211+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:31.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:31 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:31.211+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:31.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:31 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:31.211+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:31.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:31 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:31.377+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:31.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:31 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:31.377+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:31.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:31 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:31.377+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:32.189 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:31 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:31.835+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:32.189 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:31 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:31.835+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:32.189 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:31 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:31.835+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.283176+0000 front 2024-04-26T21:28:00.283405+0000 (oldest deadline 2024-04-26T21:28:24.983118+0000) 2024-04-26T21:28:32.373 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:32 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:32.062+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:32.374 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:32 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:32.062+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:32.374 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:32 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:32.062+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:32.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:32 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:32.129+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:32.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:32 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:32.129+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:32.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:31 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:31.998+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:32.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:31 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:31.998+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:32.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:31 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:31.998+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:32.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:31 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:31.998+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:32.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:32.188+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:32.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:32.188+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:32.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:32.188+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:32.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:32 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:32.373+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:32.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:32 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:32.373+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:32.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:32 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:32.373+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:33.179 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:32.846+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:33.179 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:32.846+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:33.179 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:32 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:32.846+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.283176+0000 front 2024-04-26T21:28:00.283405+0000 (oldest deadline 2024-04-26T21:28:24.983118+0000) 2024-04-26T21:28:33.339 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:33.023+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:33.339 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:33.023+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:33.339 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:33.023+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:33.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:33.100+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:33.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:33.100+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:33.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:33.100+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:33.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:32 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:32.965+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:33.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:32 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:32.966+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:33.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:32 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:32.966+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:33.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:32 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:32.966+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:33.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:33.178+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:33.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:33.178+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:33.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:33.178+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:33.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:33.338+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:33.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:33.338+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:33.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:33.338+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:33.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:33.338+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:34.198 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:33.835+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:43.980328+0000 front 2024-04-26T21:27:43.980266+0000 (oldest deadline 2024-04-26T21:28:07.480424+0000) 2024-04-26T21:28:34.198 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:33.835+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:47.981551+0000 front 2024-04-26T21:27:47.981733+0000 (oldest deadline 2024-04-26T21:28:13.881494+0000) 2024-04-26T21:28:34.198 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:33.835+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.283176+0000 front 2024-04-26T21:28:00.283405+0000 (oldest deadline 2024-04-26T21:28:24.983118+0000) 2024-04-26T21:28:34.198 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:33 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6[57532]: 2024-04-26T21:28:33.835+0000 7f86c33e0700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.884284+0000 front 2024-04-26T21:28:07.884317+0000 (oldest deadline 2024-04-26T21:28:33.184458+0000) 2024-04-26T21:28:34.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:33.980+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:34.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:33.980+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:34.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:33 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:33.980+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:34.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:34 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:34.145+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:34.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:34 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:34.145+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:34.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:34 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:34.145+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:34.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:33.981+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:34.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:33.981+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:34.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:33.981+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:34.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:33.981+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:34.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:34 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:34.197+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:34.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:34 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:34.197+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:34.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:34 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:34.197+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:34.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:34 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:34.291+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:34.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:34 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:34.291+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:34.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:34 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:34.291+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:34.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:34 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:34.291+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:34.970 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:34 smithi129 podman[91984]: 2024-04-26 21:28:34.71907486 +0000 UTC m=+5.680738564 container died 4b5a9a9c78b65a52b844cd7f29645dfd60df7a8882b75782e98804a3867a7102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T21:28:35.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:34 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:34.966+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:35.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:34 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:34.966+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:35.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:34 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:34.966+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:35.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:35 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:35.160+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:35.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:35 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:35.160+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:35.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:35 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:35.160+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:35.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:34 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:34.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:35.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:34 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:34.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:35.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:34 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:34.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:35.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:34 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:34.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:35.404 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:35 smithi129 podman[91984]: 2024-04-26 21:28:35.119835922 +0000 UTC m=+6.081499635 container cleanup 4b5a9a9c78b65a52b844cd7f29645dfd60df7a8882b75782e98804a3867a7102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T21:28:35.404 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:35 smithi129 bash[91984]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6 2024-04-26T21:28:35.405 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:35 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:35.185+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:35.405 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:35 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:35.185+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:35.405 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:35 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:35.185+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:35.405 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:35 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:35.185+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:35.719 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:35 smithi129 podman[92001]: 2024-04-26 21:28:35.404398915 +0000 UTC m=+0.689327113 container remove 4b5a9a9c78b65a52b844cd7f29645dfd60df7a8882b75782e98804a3867a7102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:28:35.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:35 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:35.317+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:35.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:35 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:35.317+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:35.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:35 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:35.317+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:35.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:35 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:35.317+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:36.186 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:35 smithi129 podman[92092]: 2024-04-26 21:28:35.800542442 +0000 UTC m=+0.017879000 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:28:36.187 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:35 smithi129 podman[92092]: 2024-04-26 21:28:35.905537559 +0000 UTC m=+0.122874117 container create eb041976ab36c2c1458f9db226a4c17e25806e55455b7c914b081007ca1137a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T21:28:36.187 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:36 smithi129 podman[92092]: 2024-04-26 21:28:36.132895484 +0000 UTC m=+0.350232042 container init eb041976ab36c2c1458f9db226a4c17e25806e55455b7c914b081007ca1137a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T21:28:36.187 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:36 smithi129 podman[92092]: 2024-04-26 21:28:36.136055803 +0000 UTC m=+0.353392362 container start eb041976ab36c2c1458f9db226a4c17e25806e55455b7c914b081007ca1137a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T21:28:36.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:35 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:35.948+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:36.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:35 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:35.948+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:36.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:35 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:35.948+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:36.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:36.183+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:36.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:36.183+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:36.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:36.183+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:36.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:36.007+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:36.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:36.007+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:36.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:36.007+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:36.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:36.007+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:36.470 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:36 smithi129 podman[92092]: 2024-04-26 21:28:36.224488372 +0000 UTC m=+0.441824939 container attach eb041976ab36c2c1458f9db226a4c17e25806e55455b7c914b081007ca1137a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:28:36.470 INFO:journalctl@ceph.osd.6.smithi129.stdout:Apr 26 21:28:36 smithi129 podman[92092]: 2024-04-26 21:28:36.425729386 +0000 UTC m=+0.643065945 container died eb041976ab36c2c1458f9db226a4c17e25806e55455b7c914b081007ca1137a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T21:28:36.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:36 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:36.185+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:36.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:36 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:36.185+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:36.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:36 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:36.185+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:36.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:36 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:36.185+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:36.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:36 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:36.337+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:36.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:36 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:36.337+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:36.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:36 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:36.337+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:36.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:36 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:36.337+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:37.077 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T21:28:37.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:28:37.115 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T21:28:37.115 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T21:28:37.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.7 2024-04-26T21:28:37.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:36 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:36.921+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:37.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:36 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:36.921+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:37.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:36 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:36.921+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:37.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:37 smithi150 systemd[1]: Stopping Ceph osd.7 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:28:37.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:37.175+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:37.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:37.175+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:37.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:37.175+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:37.384 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:36.974+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:37.384 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:36.974+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:37.384 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:36.974+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:37.384 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:36 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:36.974+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:37.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:37 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:37.143+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:37.519 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:37 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:37.143+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:37.519 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:37 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:37.143+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:37.520 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:37 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:37.143+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:37.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:37 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:37.294+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:37.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:37 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:37.294+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:37.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:37 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:37.294+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:37.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:37 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:37.294+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:37.882 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:37.429+0000 7f3ec763f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:28:37.883 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:37.429+0000 7f3ec763f700 -1 osd.7 91 *** Got signal Terminated *** 2024-04-26T21:28:37.883 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:37.429+0000 7f3ec763f700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:28:38.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:37 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:37.902+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:38.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:37 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:37.902+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:38.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:37 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:37.902+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:38.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:38 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:38.215+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:38.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:38 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:38.215+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:38.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:38 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:38.215+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:38.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:37.971+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:38.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:37.971+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:38.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:37.971+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:38.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:37.971+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:38.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:38 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:38.192+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:38.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:38 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:38.192+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:38.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:38 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:38.192+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:38.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:38 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:38.192+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:38.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:38 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:38.297+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:38.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:38 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:38.297+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:38.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:38 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:38.297+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:38.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:38 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:38.297+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:39.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:38 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:38.910+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:39.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:38 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:38.910+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:39.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:38 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:38.910+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:39.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:39.195+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:39.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:39.195+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:39.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:39.195+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:39.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:39.015+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:39.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:39.015+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:39.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:39.015+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:39.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:39.015+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:39.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:39 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:39.215+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:39.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:39 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:39.215+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:39.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:39 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:39.215+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:39.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:39 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:39.215+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:39.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:39 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:39.290+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:39.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:39 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:39.290+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:39.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:39 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:39.290+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:39.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:39 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:39.290+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:40.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:39 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:39.861+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:40.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:39 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:39.861+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:40.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:39 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:39.861+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:40.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:39 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:39.861+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:40.244 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:39.987+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:40.244 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:39.987+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:40.244 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:39.987+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:40.245 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:39.987+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:40.245 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:39 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:39.987+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:40.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:40 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:40.243+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:40.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:40 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:40.243+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:40.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:40 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:40.243+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:40.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:40 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:40.243+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:15.518080+0000 front 2024-04-26T21:28:15.518102+0000 (oldest deadline 2024-04-26T21:28:39.618173+0000) 2024-04-26T21:28:40.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:40 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:40.247+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:40.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:40 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:40.247+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:40.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:40 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:40.247+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:40.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:40 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:40.247+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:40.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:40 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:40.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:40.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:40 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:40.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:40.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:40 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:40.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:40.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:40 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:40.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:41.227 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:40 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:40.955+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:41.228 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:40 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:40.955+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:41.228 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:40 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:40.955+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:41.228 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:40 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:40.955+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:41.228 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:40 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:40.955+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:41.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:40 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:40.874+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:41.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:40 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:40.874+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:41.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:40 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:40.874+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:41.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:40 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:40.874+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:41.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:41 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:41.205+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:41.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:41 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:41.205+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:41.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:41 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:41.205+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:41.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:41 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:41.205+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:41.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:41 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:41.226+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:41.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:41 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:41.226+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:41.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:41 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:41.226+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:41.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:41 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:41.226+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:15.518080+0000 front 2024-04-26T21:28:15.518102+0000 (oldest deadline 2024-04-26T21:28:39.618173+0000) 2024-04-26T21:28:41.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:41 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:41.278+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:41.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:41 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:41.278+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:41.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:41 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:41.278+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:41.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:41 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:41.278+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:42.227 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:41 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:41.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:42.227 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:41 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:41.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:42.227 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:41 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:41.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:42.228 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:41 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:41.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:42.228 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:41 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:41.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:42.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:41 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:41.901+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:42.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:41 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:41.901+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:42.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:41 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:41.901+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:42.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:41 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:41.901+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:42.558 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:42 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:42.229+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:42.559 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:42 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:42.229+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:42.559 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:42 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:42.229+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:42.559 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:42 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:42.229+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:42.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:42.227+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:50.111746+0000 front 2024-04-26T21:27:50.111776+0000 (oldest deadline 2024-04-26T21:28:14.812077+0000) 2024-04-26T21:28:42.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:42.227+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.614059+0000 front 2024-04-26T21:28:00.613875+0000 (oldest deadline 2024-04-26T21:28:22.313996+0000) 2024-04-26T21:28:42.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:42.227+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.317138+0000 front 2024-04-26T21:28:07.317071+0000 (oldest deadline 2024-04-26T21:28:32.617032+0000) 2024-04-26T21:28:42.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7[50374]: 2024-04-26T21:28:42.227+0000 7f3ebfc2b700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:15.518080+0000 front 2024-04-26T21:28:15.518102+0000 (oldest deadline 2024-04-26T21:28:39.618173+0000) 2024-04-26T21:28:42.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:42 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:42.315+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:42.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:42 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:42.315+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:42.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:42 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:42.315+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:42.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:42 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:42.315+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:43.056 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:42 smithi150 podman[67233]: 2024-04-26 21:28:42.78177506 +0000 UTC m=+5.474094626 container died 93d420dc3a1d5ccc1290dc6dd920263fb4f48c959e73e413a2232bedb3ccf9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:28:43.056 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:42.980+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:43.056 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:42.980+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:43.056 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:42.980+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:43.057 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:42.980+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:43.057 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:42.980+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:43.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:42 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:42.910+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:43.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:42 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:42.910+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:43.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:42 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:42.910+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:43.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:42 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:42.910+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:43.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:43 smithi150 podman[67233]: 2024-04-26 21:28:43.146741476 +0000 UTC m=+5.839061047 container cleanup 93d420dc3a1d5ccc1290dc6dd920263fb4f48c959e73e413a2232bedb3ccf9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T21:28:43.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:43 smithi150 bash[67233]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7 2024-04-26T21:28:43.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:43 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:43.219+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:43.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:43 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:43.219+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:43.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:43 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:43.219+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:43.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:43 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:43.219+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:43.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:43 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:43.219+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:43.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:43 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:43.324+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:43.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:43 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:43.324+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:43.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:43 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:43.324+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:43.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:43 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:43.324+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:43.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:43 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:43.324+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:43.768 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:43 smithi150 podman[67250]: 2024-04-26 21:28:43.452395299 +0000 UTC m=+0.668775282 container remove 93d420dc3a1d5ccc1290dc6dd920263fb4f48c959e73e413a2232bedb3ccf9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:28:44.068 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:43 smithi150 podman[67339]: 2024-04-26 21:28:43.671022186 +0000 UTC m=+0.018019458 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:28:44.069 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:43 smithi150 podman[67339]: 2024-04-26 21:28:43.798827452 +0000 UTC m=+0.145824713 container create e77f85cc28a918cc6b08c730e5065eca89de8ef3e23ab770df5aef4853b1a088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T21:28:44.069 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:43 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:43.946+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:44.069 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:43 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:43.946+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:44.069 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:43 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:43.946+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:44.069 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:43 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:43.946+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:44.069 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:43 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:43.946+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:44.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:43 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:43.919+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:44.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:43 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:43.919+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:44.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:43 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:43.919+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:44.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:43 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:43.919+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:44.346 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:44 smithi150 podman[67339]: 2024-04-26 21:28:44.068379682 +0000 UTC m=+0.415376945 container init e77f85cc28a918cc6b08c730e5065eca89de8ef3e23ab770df5aef4853b1a088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T21:28:44.347 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:44 smithi150 podman[67339]: 2024-04-26 21:28:44.072330812 +0000 UTC m=+0.419328075 container start e77f85cc28a918cc6b08c730e5065eca89de8ef3e23ab770df5aef4853b1a088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T21:28:44.347 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:44 smithi150 podman[67339]: 2024-04-26 21:28:44.168282857 +0000 UTC m=+0.515280121 container attach e77f85cc28a918cc6b08c730e5065eca89de8ef3e23ab770df5aef4853b1a088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T21:28:44.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 26 21:28:44 smithi150 podman[67339]: 2024-04-26 21:28:44.377400086 +0000 UTC m=+0.724397341 container died e77f85cc28a918cc6b08c730e5065eca89de8ef3e23ab770df5aef4853b1a088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T21:28:44.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:44 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:44.241+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:44.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:44 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:44.241+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:44.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:44 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:44.241+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:44.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:44 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:44.241+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:44.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:44 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:44.241+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:44.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:44.369+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:44.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:44.369+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:44.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:44.369+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:44.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:44.369+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:44.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:44.369+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:45.006 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:44 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:44.964+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:45.006 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:44 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:44.965+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:45.006 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:44 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:44.965+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:45.006 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:44 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:44.965+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:45.006 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:44 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:44.965+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:45.028 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T21:28:45.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:28:45.066 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T21:28:45.066 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T21:28:45.066 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.8 2024-04-26T21:28:45.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:44.892+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:45.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:44.892+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:45.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:44.892+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:45.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:44 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:44.892+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:45.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:45 smithi171 systemd[1]: Stopping Ceph osd.8 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:28:45.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:45 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:45.271+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:45.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:45 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:45.271+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:45.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:45 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:45.271+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:45.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:45 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:45.271+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:45.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:45 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:45.271+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:45.741 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:45.384+0000 7fe46b81a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:28:45.741 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:45.384+0000 7fe46b81a700 -1 osd.8 91 *** Got signal Terminated *** 2024-04-26T21:28:45.741 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:45.384+0000 7fe46b81a700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:28:45.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:45.412+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:45.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:45.412+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:45.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:45.412+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:45.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:45.412+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:45.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:45.412+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:46.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:45.911+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:46.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:45.911+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:46.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:45.911+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:46.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:45 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:45.911+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:46.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:45 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:45.919+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:46.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:45 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:45.919+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:46.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:45 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:45.919+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:46.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:45 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:45.919+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:46.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:45 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:45.919+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:46.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:46 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:46.225+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:46.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:46 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:46.225+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:46.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:46 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:46.225+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:46.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:46 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:46.225+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:46.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:46 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:46.225+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:46.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:46 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:46.420+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:46.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:46 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:46.420+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:46.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:46 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:46.420+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:46.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:46 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:46.420+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:46.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:46 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:46.420+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:47.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:46 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:46.959+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:47.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:46 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:46.959+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:47.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:46 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:46.959+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:47.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:46 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:46.959+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:47.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:46 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:46.944+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:47.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:46 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:46.944+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:47.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:46 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:46.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:47.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:46 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:46.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:47.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:46 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:46.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:47.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:47 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:47.252+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:47.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:47 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:47.252+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:47.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:47 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:47.252+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:47.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:47 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:47.252+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:47.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:47 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:47.252+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:47.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:47.450+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:47.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:47.450+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:47.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:47.450+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:47.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:47.450+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:47.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:47.450+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:48.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:47.970+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:48.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:47.970+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:48.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:47.970+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:48.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:47.970+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:48.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:47 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:47.970+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.958172+0000 front 2024-04-26T21:28:24.958141+0000 (oldest deadline 2024-04-26T21:28:47.258144+0000) 2024-04-26T21:28:48.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:47 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:47.898+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:48.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:47 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:47.898+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:48.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:47 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:47.898+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:48.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:47 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:47.898+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:48.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:47 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:47.898+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:48.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:47 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:47.898+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:48.521 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:48.407+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:48.521 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:48.407+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:48.521 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:48.407+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:48.521 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:48.407+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:48.521 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:48.407+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:48.521 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:48.407+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:48.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:48.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:48.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:48.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:48.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:48.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:48.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:48.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:48.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:48 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:48.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:49.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:48.938+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:49.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:48.938+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:49.241 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:48.938+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:49.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:48.938+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:49.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:48 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:48.938+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.958172+0000 front 2024-04-26T21:28:24.958141+0000 (oldest deadline 2024-04-26T21:28:47.258144+0000) 2024-04-26T21:28:49.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:48.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:49.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:48.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:49.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:48.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:49.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:48.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:49.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:48.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:49.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:48.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:49.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:49 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:49.300+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:49.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:49 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:49.300+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:49.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:49 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:49.300+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:49.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:49 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:49.300+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:49.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:49 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:49.300+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:49.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:49.392+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:49.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:49.392+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:49.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:49.392+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:49.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:49.392+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:49.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:49.392+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:49.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:49.392+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:49.957 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:49.954+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.852613+0000 front 2024-04-26T21:27:51.852503+0000 (oldest deadline 2024-04-26T21:28:16.453154+0000) 2024-04-26T21:28:49.957 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:49.954+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.554042+0000 front 2024-04-26T21:27:57.554177+0000 (oldest deadline 2024-04-26T21:28:22.854227+0000) 2024-04-26T21:28:49.957 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:49.954+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.755300+0000 front 2024-04-26T21:28:08.755412+0000 (oldest deadline 2024-04-26T21:28:31.655290+0000) 2024-04-26T21:28:49.958 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:49.954+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:14.456800+0000 front 2024-04-26T21:28:14.456841+0000 (oldest deadline 2024-04-26T21:28:39.756714+0000) 2024-04-26T21:28:49.958 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:49 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8[52228]: 2024-04-26T21:28:49.954+0000 7fe463e06700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.958172+0000 front 2024-04-26T21:28:24.958141+0000 (oldest deadline 2024-04-26T21:28:47.258144+0000) 2024-04-26T21:28:50.382 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:49 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:49.904+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:50.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:49 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:49.905+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:50.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:49 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:49.905+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:50.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:49 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:49.905+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:50.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:49 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:49.905+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:50.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:49 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:49.905+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:50.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:50 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:50.267+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:50.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:50 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:50.267+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:50.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:50 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:50.267+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:50.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:50 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:50.267+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:50.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:50 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:50.267+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:50.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:50 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:50.402+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:50.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:50 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:50.402+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:50.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:50 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:50.402+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:50.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:50 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:50.402+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:50.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:50 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:50.402+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:50.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:50 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:50.402+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:51.101 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:50 smithi171 podman[69174]: 2024-04-26 21:28:50.791361693 +0000 UTC m=+5.530234691 container died c3c530c177c74b3d4879eb27caad13b71f8d67834ce00e1d9c898e0d55e81ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T21:28:51.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:50 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:50.916+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:51.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:50 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:50.916+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:51.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:50 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:50.916+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:51.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:50 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:50.916+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:51.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:50 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:50.916+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:51.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:50 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:50.916+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:51.384 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:51 smithi171 podman[69174]: 2024-04-26 21:28:51.101175063 +0000 UTC m=+5.840048048 container cleanup c3c530c177c74b3d4879eb27caad13b71f8d67834ce00e1d9c898e0d55e81ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:28:51.384 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:51 smithi171 bash[69174]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8 2024-04-26T21:28:51.384 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:51 smithi171 podman[69204]: 2024-04-26 21:28:51.319222144 +0000 UTC m=+0.521621725 container remove c3c530c177c74b3d4879eb27caad13b71f8d67834ce00e1d9c898e0d55e81ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T21:28:51.385 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:51 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:51.382+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:51.385 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:51 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:51.382+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:51.385 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:51 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:51.382+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:51.385 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:51 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:51.382+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:51.385 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:51 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:51.382+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:51.385 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:51 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:51.382+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:51.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:51.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:51.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:51.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:51.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:51.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:51.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:51.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:51.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:51.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:51.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:51 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:51.297+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.570952+0000 front 2024-04-26T21:28:22.570850+0000 (oldest deadline 2024-04-26T21:28:50.771190+0000) 2024-04-26T21:28:51.991 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:51 smithi171 podman[69293]: 2024-04-26 21:28:51.623949011 +0000 UTC m=+0.017529773 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:28:51.991 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:51 smithi171 podman[69293]: 2024-04-26 21:28:51.744879001 +0000 UTC m=+0.138459753 container create 546ea8cc7fb92118c6bd4db1b9bf58210413d7747b2ac51f146cfdc0a92464c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T21:28:52.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:51 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:51.867+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:52.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:51 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:51.867+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:52.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:51 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:51.867+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:52.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:51 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:51.867+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:52.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:51 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:51.867+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:52.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:51 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:51.867+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:52.324 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:52 smithi171 podman[69293]: 2024-04-26 21:28:52.031476385 +0000 UTC m=+0.425057142 container init 546ea8cc7fb92118c6bd4db1b9bf58210413d7747b2ac51f146cfdc0a92464c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T21:28:52.324 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:52 smithi171 podman[69293]: 2024-04-26 21:28:52.034610817 +0000 UTC m=+0.428191571 container start 546ea8cc7fb92118c6bd4db1b9bf58210413d7747b2ac51f146cfdc0a92464c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T21:28:52.324 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:52 smithi171 podman[69293]: 2024-04-26 21:28:52.106311562 +0000 UTC m=+0.499892333 container attach 546ea8cc7fb92118c6bd4db1b9bf58210413d7747b2ac51f146cfdc0a92464c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T21:28:52.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:52.283+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:52.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:52.283+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:52.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:52.283+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:52.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:52.283+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:52.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:52.283+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:52.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:52 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:52.283+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.570952+0000 front 2024-04-26T21:28:22.570850+0000 (oldest deadline 2024-04-26T21:28:50.771190+0000) 2024-04-26T21:28:52.592 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 26 21:28:52 smithi171 podman[69293]: 2024-04-26 21:28:52.323871051 +0000 UTC m=+0.717451812 container died 546ea8cc7fb92118c6bd4db1b9bf58210413d7747b2ac51f146cfdc0a92464c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T21:28:52.593 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:52 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:52.349+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:52.593 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:52 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:52.349+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:52.593 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:52 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:52.349+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:52.593 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:52 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:52.349+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:52.593 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:52 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:52.349+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:52.593 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:52 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:52.349+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:52.937 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T21:28:52.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:28:52.976 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T21:28:52.977 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T21:28:52.977 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.9 2024-04-26T21:28:53.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:52 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:52.829+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:53.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:52 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:52.829+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:53.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:52 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:52.829+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:53.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:52 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:52.829+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:53.134 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:52 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:52.829+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:53.134 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:52 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:52.829+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:53.264 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 systemd[1]: Stopping Ceph osd.9 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:28:53.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:53.262+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:53.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:53.262+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:53.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:53.262+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:53.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:53.262+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:53.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:53.262+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:53.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:53.262+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.570952+0000 front 2024-04-26T21:28:22.570850+0000 (oldest deadline 2024-04-26T21:28:50.771190+0000) 2024-04-26T21:28:53.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:53.305+0000 7fb0606bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:28:53.721 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:53.305+0000 7fb0606bb700 -1 osd.9 91 *** Got signal Terminated *** 2024-04-26T21:28:53.721 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:53 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:53.305+0000 7fb0606bb700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:28:53.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:53 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:53.301+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:53.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:53 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:53.301+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:53.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:53 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:53.301+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:53.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:53 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:53.301+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:53.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:53 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:53.301+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:53.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:53 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:53.301+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:54.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:53 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:53.855+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:54.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:53 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:53.855+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:54.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:53 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:53.855+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:54.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:53 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:53.855+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:54.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:53 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:53.855+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:54.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:53 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:53.855+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:54.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:54 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:54.274+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:54.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:54 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:54.274+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:54.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:54 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:54.274+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:54.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:54 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:54.274+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:54.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:54 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:54.274+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:54.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:54 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:54.274+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.570952+0000 front 2024-04-26T21:28:22.570850+0000 (oldest deadline 2024-04-26T21:28:50.771190+0000) 2024-04-26T21:28:54.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:54 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:54.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:54.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:54 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:54.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:54.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:54 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:54.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:54.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:54 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:54.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:54.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:54 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:54.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:54.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:54 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:54.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:55.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:54 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:54.877+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:55.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:54 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:54.877+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:55.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:54 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:54.877+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:55.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:54 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:54.877+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:55.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:54 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:54.877+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:55.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:54 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:54.877+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:55.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:55 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:55.248+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:55.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:55 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:55.248+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:55.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:55 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:55.248+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:55.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:55 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:55.248+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:55.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:55 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:55.248+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:55.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:55 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:55.248+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.570952+0000 front 2024-04-26T21:28:22.570850+0000 (oldest deadline 2024-04-26T21:28:50.771190+0000) 2024-04-26T21:28:55.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:55 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:55.305+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:55.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:55 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:55.305+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:55.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:55 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:55.305+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:55.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:55 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:55.305+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:55.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:55 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:55.305+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:55.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:55 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:55.305+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:56.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:55 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:55.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:56.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:55 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:55.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:56.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:55 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:55.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:56.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:55 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:55.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:56.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:55 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:55.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:56.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:55 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:55.901+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:56.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:56 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:56.284+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:56.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:56 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:56.284+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:56.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:56 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:56.284+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:56.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:56 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:56.284+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:56.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:56 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:56.284+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:56.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:56 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:56.284+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.570952+0000 front 2024-04-26T21:28:22.570850+0000 (oldest deadline 2024-04-26T21:28:50.771190+0000) 2024-04-26T21:28:56.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:56 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:56.284+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:30.771589+0000 front 2024-04-26T21:28:30.771627+0000 (oldest deadline 2024-04-26T21:28:55.471669+0000) 2024-04-26T21:28:56.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:56 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:56.258+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:56.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:56 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:56.258+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:56.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:56 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:56.258+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:56.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:56 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:56.258+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:56.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:56 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:56.258+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:56.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:56 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:56.258+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:57.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:56 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:56.894+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:57.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:56 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:56.894+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:57.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:56 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:56.894+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:57.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:56 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:56.894+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:57.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:56 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:56.894+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:57.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:56 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:56.894+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:57.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:57 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:57.285+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.863422+0000 front 2024-04-26T21:27:41.863453+0000 (oldest deadline 2024-04-26T21:28:06.563645+0000) 2024-04-26T21:28:57.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:57 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:57.285+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:52.865939+0000 front 2024-04-26T21:27:52.865997+0000 (oldest deadline 2024-04-26T21:28:15.765749+0000) 2024-04-26T21:28:57.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:57 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:57.285+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:01.066961+0000 front 2024-04-26T21:28:01.067066+0000 (oldest deadline 2024-04-26T21:28:24.567064+0000) 2024-04-26T21:28:57.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:57 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:57.285+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:08.469138+0000 front 2024-04-26T21:28:08.469205+0000 (oldest deadline 2024-04-26T21:28:34.369196+0000) 2024-04-26T21:28:57.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:57 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:57.285+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.270554+0000 front 2024-04-26T21:28:17.270416+0000 (oldest deadline 2024-04-26T21:28:42.570230+0000) 2024-04-26T21:28:57.720 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:57 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:57.285+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.570952+0000 front 2024-04-26T21:28:22.570850+0000 (oldest deadline 2024-04-26T21:28:50.771190+0000) 2024-04-26T21:28:57.721 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:57 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9[64144]: 2024-04-26T21:28:57.285+0000 7fb058ca7700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:30.771589+0000 front 2024-04-26T21:28:30.771627+0000 (oldest deadline 2024-04-26T21:28:55.471669+0000) 2024-04-26T21:28:57.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:57 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:57.248+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:57.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:57 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:57.248+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:57.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:57 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:57.248+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:57.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:57 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:57.248+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:57.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:57 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:57.248+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:57.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:57 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:57.248+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:58.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:57 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:57.925+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:58.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:57 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:57.925+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:58.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:57 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:57.925+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:58.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:57 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:57.925+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:58.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:57 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:57.925+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:58.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:57 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:57.925+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:58.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:58 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:58.296+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:58.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:58 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:58.296+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:58.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:58 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:58.296+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:58.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:58 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:58.296+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:58.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:58 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:58.296+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:58.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:58 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:58.296+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:28:59.171 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:58 smithi129 podman[92210]: 2024-04-26 21:28:58.838887875 +0000 UTC m=+5.666012864 container died 4f05870c9bef8dbbac78167b4020cbcb7f3c10a206d6eb0ce68ffd389f1d6305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T21:28:59.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:58 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:58.944+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:28:59.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:58 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:58.944+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:28:59.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:58 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:58.944+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:28:59.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:58 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:58.944+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:28:59.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:58 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:58.944+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:28:59.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:58 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:58.944+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:28:59.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:58 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:58.944+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:33.005141+0000 front 2024-04-26T21:28:33.005109+0000 (oldest deadline 2024-04-26T21:28:58.905077+0000) 2024-04-26T21:28:59.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:59 smithi129 podman[92210]: 2024-04-26 21:28:59.171649338 +0000 UTC m=+5.998774333 container cleanup 4f05870c9bef8dbbac78167b4020cbcb7f3c10a206d6eb0ce68ffd389f1d6305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:28:59.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:59 smithi129 bash[92210]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9 2024-04-26T21:28:59.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:59 smithi129 podman[92228]: 2024-04-26 21:28:59.356228372 +0000 UTC m=+0.513330035 container remove 4f05870c9bef8dbbac78167b4020cbcb7f3c10a206d6eb0ce68ffd389f1d6305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-26T21:28:59.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:59 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:59.261+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:28:59.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:59 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:59.261+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:28:59.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:59 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:59.261+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:28:59.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:59 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:59.261+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:28:59.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:59 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:59.261+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:28:59.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:28:59 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:28:59.261+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:00.067 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:59 smithi129 podman[92317]: 2024-04-26 21:28:59.66657042 +0000 UTC m=+0.017854845 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:29:00.068 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:59 smithi129 podman[92317]: 2024-04-26 21:28:59.765165184 +0000 UTC m=+0.116449608 container create e818f27d179b87d4d2e76668eb84406e9faecd2414f63d158b3895c3a9c2a44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T21:29:00.068 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:59 smithi129 podman[92317]: 2024-04-26 21:28:59.992510809 +0000 UTC m=+0.343795243 container init e818f27d179b87d4d2e76668eb84406e9faecd2414f63d158b3895c3a9c2a44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T21:29:00.068 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:28:59 smithi129 podman[92317]: 2024-04-26 21:28:59.995645428 +0000 UTC m=+0.346929850 container start e818f27d179b87d4d2e76668eb84406e9faecd2414f63d158b3895c3a9c2a44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:29:00.068 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:29:00 smithi129 podman[92317]: 2024-04-26 21:29:00.067307895 +0000 UTC m=+0.418592316 container attach e818f27d179b87d4d2e76668eb84406e9faecd2414f63d158b3895c3a9c2a44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T21:29:00.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:59 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:59.909+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:29:00.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:59 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:59.909+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:29:00.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:59 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:59.909+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:29:00.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:59 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:59.909+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:29:00.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:59 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:59.909+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:29:00.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:59 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:59.909+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:29:00.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:28:59 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:28:59.909+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:33.005141+0000 front 2024-04-26T21:28:33.005109+0000 (oldest deadline 2024-04-26T21:28:58.905077+0000) 2024-04-26T21:29:00.470 INFO:journalctl@ceph.osd.9.smithi129.stdout:Apr 26 21:29:00 smithi129 podman[92317]: 2024-04-26 21:29:00.281844659 +0000 UTC m=+0.633129081 container died e818f27d179b87d4d2e76668eb84406e9faecd2414f63d158b3895c3a9c2a44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:29:00.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:00 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:00.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:00.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:00 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:00.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:00.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:00 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:00.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:00.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:00 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:00.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:00.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:00 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:00.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:00.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:00 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:00.281+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:00.857 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T21:29:00.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:00.898 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T21:29:00.898 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T21:29:00.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.10 2024-04-26T21:29:01.209 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:00 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:00.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:29:01.209 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:00 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:00.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:29:01.209 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:00 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:00.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:29:01.209 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:00 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:00.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:29:01.209 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:00 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:00.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:29:01.210 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:00 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:00.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:29:01.210 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:00 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:00.945+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:33.005141+0000 front 2024-04-26T21:28:33.005109+0000 (oldest deadline 2024-04-26T21:28:58.905077+0000) 2024-04-26T21:29:01.210 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:00 smithi150 systemd[1]: Stopping Ceph osd.10 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:01.210 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.207+0000 7f1e632e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:01.210 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.208+0000 7f1e632e8700 -1 osd.10 91 *** Got signal Terminated *** 2024-04-26T21:29:01.210 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.208+0000 7f1e632e8700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:29:01.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:01 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:01.254+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:01.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:01 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:01.254+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:01.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:01 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:01.254+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:01.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:01 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:01.254+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:01.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:01 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:01.254+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:01.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:01 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:01.254+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:02.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:29:02.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:29:02.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:29:02.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:29:02.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:29:02.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:29:02.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:01 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:01.992+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:33.005141+0000 front 2024-04-26T21:28:33.005109+0000 (oldest deadline 2024-04-26T21:28:58.905077+0000) 2024-04-26T21:29:02.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:02 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:02.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:02.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:02 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:02.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:02.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:02 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:02.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:02.741 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:02 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:02.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:02.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:02 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:02.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:02.742 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:02 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:02.269+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:03.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:02 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:02.943+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:29:03.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:02 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:02.943+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:29:03.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:02 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:02.943+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:29:03.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:02 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:02.943+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:29:03.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:02 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:02.943+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:29:03.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:02 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:02.943+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:29:03.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:02 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:02.943+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:33.005141+0000 front 2024-04-26T21:28:33.005109+0000 (oldest deadline 2024-04-26T21:28:58.905077+0000) 2024-04-26T21:29:03.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:03 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:03.237+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:03.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:03 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:03.237+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:03.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:03 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:03.237+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:03.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:03 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:03.237+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:03.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:03 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:03.237+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:03.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:03 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:03.237+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:04.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:03 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:03.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:29:04.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:03 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:03.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:29:04.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:03 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:03.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:29:04.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:03 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:03.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:29:04.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:03 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:03.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:29:04.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:03 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:03.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:29:04.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:03 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:03.936+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:33.005141+0000 front 2024-04-26T21:28:33.005109+0000 (oldest deadline 2024-04-26T21:28:58.905077+0000) 2024-04-26T21:29:04.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:04 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:04.217+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:04.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:04 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:04.217+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:04.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:04 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:04.217+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:04.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:04 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:04.217+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:04.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:04 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:04.217+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:04.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:04 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:04.217+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:04.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:04 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:04.217+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:05.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:04 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:04.983+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:29:05.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:04 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:04.983+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:29:05.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:04 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:04.983+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:29:05.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:04 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:04.983+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:29:05.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:04 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:04.983+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:29:05.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:04 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:04.983+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:29:05.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:04 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:04.983+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:33.005141+0000 front 2024-04-26T21:28:33.005109+0000 (oldest deadline 2024-04-26T21:28:58.905077+0000) 2024-04-26T21:29:05.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:05 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:05.194+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:05.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:05 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:05.194+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:05.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:05 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:05.194+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:05.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:05 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:05.194+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:05.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:05 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:05.194+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:05.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:05 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:05.194+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:05.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:05 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:05.194+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:06.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:06.032+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:44.495419+0000 front 2024-04-26T21:27:44.495401+0000 (oldest deadline 2024-04-26T21:28:06.795376+0000) 2024-04-26T21:29:06.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:06.032+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.496446+0000 front 2024-04-26T21:27:51.496472+0000 (oldest deadline 2024-04-26T21:28:15.596682+0000) 2024-04-26T21:29:06.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:06.032+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:28:00.298054+0000 front 2024-04-26T21:28:00.297925+0000 (oldest deadline 2024-04-26T21:28:23.197985+0000) 2024-04-26T21:29:06.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:06.032+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.199731+0000 front 2024-04-26T21:28:07.199768+0000 (oldest deadline 2024-04-26T21:28:31.899581+0000) 2024-04-26T21:29:06.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:06.032+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:17.601465+0000 front 2024-04-26T21:28:17.601489+0000 (oldest deadline 2024-04-26T21:28:39.901561+0000) 2024-04-26T21:29:06.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:06.032+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:24.602704+0000 front 2024-04-26T21:28:24.602615+0000 (oldest deadline 2024-04-26T21:28:47.403030+0000) 2024-04-26T21:29:06.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-26T21:29:06.032+0000 7f1e5f8dc700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-04-26T21:28:33.005141+0000 front 2024-04-26T21:28:33.005109+0000 (oldest deadline 2024-04-26T21:28:58.905077+0000) 2024-04-26T21:29:06.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:06.151+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:06.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:06.151+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:06.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:06.151+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:06.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:06.151+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:06.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:06.151+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:06.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:06.151+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:06.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:06 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:06.151+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:06.882 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 podman[67458]: 2024-04-26 21:29:06.612633454 +0000 UTC m=+5.510871585 container died 590ee2f02365d7c9851ae5bfc331784b5c8e54c2e51e9fa5f7c24b46efd03dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T21:29:07.269 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 podman[67458]: 2024-04-26 21:29:06.979503857 +0000 UTC m=+5.877741974 container cleanup 590ee2f02365d7c9851ae5bfc331784b5c8e54c2e51e9fa5f7c24b46efd03dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T21:29:07.269 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:06 smithi150 bash[67458]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10 2024-04-26T21:29:07.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:07.122+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:07.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:07.122+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:07.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:07.122+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:07.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:07.122+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:07.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:07.122+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:07.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:07.122+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:07.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:07 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:07.122+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:07.599 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:07 smithi150 podman[67475]: 2024-04-26 21:29:07.269097943 +0000 UTC m=+0.651210911 container remove 590ee2f02365d7c9851ae5bfc331784b5c8e54c2e51e9fa5f7c24b46efd03dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T21:29:07.866 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:07 smithi150 podman[67564]: 2024-04-26 21:29:07.502232586 +0000 UTC m=+0.017909561 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:29:07.866 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:07 smithi150 podman[67564]: 2024-04-26 21:29:07.620675637 +0000 UTC m=+0.136352584 container create ab9853455fe17b304709404aaa58578b537c3ce99375bf56c5c5c33ba14dc09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:29:07.866 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:07 smithi150 podman[67564]: 2024-04-26 21:29:07.865307645 +0000 UTC m=+0.380984607 container init ab9853455fe17b304709404aaa58578b537c3ce99375bf56c5c5c33ba14dc09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T21:29:08.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:07 smithi150 podman[67564]: 2024-04-26 21:29:07.868603567 +0000 UTC m=+0.384280515 container start ab9853455fe17b304709404aaa58578b537c3ce99375bf56c5c5c33ba14dc09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T21:29:08.133 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:07 smithi150 podman[67564]: 2024-04-26 21:29:07.947939335 +0000 UTC m=+0.463616300 container attach ab9853455fe17b304709404aaa58578b537c3ce99375bf56c5c5c33ba14dc09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T21:29:08.459 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 26 21:29:08 smithi150 podman[67564]: 2024-04-26 21:29:08.15939729 +0000 UTC m=+0.675074255 container died ab9853455fe17b304709404aaa58578b537c3ce99375bf56c5c5c33ba14dc09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T21:29:08.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:08.148+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:08.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:08.148+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:08.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:08.148+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:08.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:08.148+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:08.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:08.148+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:08.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:08.148+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:08.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:08 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:08.148+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:08.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T21:29:08.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:08.836 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T21:29:08.836 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T21:29:08.836 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.11 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:08 smithi171 systemd[1]: Stopping Ceph osd.11 for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.107+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.107+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.107+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.107+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.107+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.107+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.107+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.161+0000 7ff70bde4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:09.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.161+0000 7ff70bde4700 -1 osd.11 91 *** Got signal Terminated *** 2024-04-26T21:29:09.164 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:09 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:09.161+0000 7ff70bde4700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:29:10.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:10.070+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:10.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:10.070+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:10.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:10.070+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:10.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:10.070+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:10.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:10.070+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:10.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:10.070+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:10.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:10 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:10.070+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:11.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:11.113+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:11.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:11.113+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:11.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:11.113+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:11.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:11.113+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:11.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:11.113+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:11.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:11.113+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:11.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:11 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:11.113+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:12.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:12.132+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:12.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:12.132+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:12.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:12.132+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:12.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:12.132+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:12.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:12.132+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:12.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:12.132+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:12.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:12.132+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:12.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:12 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:12.132+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-26T21:28:49.408670+0000 front 2024-04-26T21:28:49.408631+0000 (oldest deadline 2024-04-26T21:29:11.708656+0000) 2024-04-26T21:29:13.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:13.162+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-04-26T21:27:41.401539+0000 front 2024-04-26T21:27:41.401713+0000 (oldest deadline 2024-04-26T21:28:06.101719+0000) 2024-04-26T21:29:13.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:13.162+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-26T21:27:51.803868+0000 front 2024-04-26T21:27:51.803620+0000 (oldest deadline 2024-04-26T21:28:17.103756+0000) 2024-04-26T21:29:13.491 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:13.162+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-26T21:27:57.104164+0000 front 2024-04-26T21:27:57.104082+0000 (oldest deadline 2024-04-26T21:28:23.004137+0000) 2024-04-26T21:29:13.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:13.162+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-04-26T21:28:07.705048+0000 front 2024-04-26T21:28:07.705214+0000 (oldest deadline 2024-04-26T21:28:32.404929+0000) 2024-04-26T21:29:13.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:13.162+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-26T21:28:12.405479+0000 front 2024-04-26T21:28:12.405331+0000 (oldest deadline 2024-04-26T21:28:42.405705+0000) 2024-04-26T21:29:13.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:13.162+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-26T21:28:22.406209+0000 front 2024-04-26T21:28:22.406236+0000 (oldest deadline 2024-04-26T21:28:48.306168+0000) 2024-04-26T21:29:13.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:13.162+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-26T21:28:39.408047+0000 front 2024-04-26T21:28:39.408116+0000 (oldest deadline 2024-04-26T21:29:03.507988+0000) 2024-04-26T21:29:13.492 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:13 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11[59002]: 2024-04-26T21:29:13.162+0000 7ff7083d8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-26T21:28:49.408670+0000 front 2024-04-26T21:28:49.408631+0000 (oldest deadline 2024-04-26T21:29:11.708656+0000) 2024-04-26T21:29:14.844 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:14 smithi171 podman[69397]: 2024-04-26 21:29:14.563340922 +0000 UTC m=+5.531094015 container died e4c55b05fe60a3bfc9eab5e7150544ecfeaa77bd800c2c247429cd3aceed9a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T21:29:14.844 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:14 smithi171 podman[69397]: 2024-04-26 21:29:14.844568477 +0000 UTC m=+5.812321565 container cleanup e4c55b05fe60a3bfc9eab5e7150544ecfeaa77bd800c2c247429cd3aceed9a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T21:29:15.095 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:14 smithi171 bash[69397]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11 2024-04-26T21:29:15.428 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:15 smithi171 podman[69414]: 2024-04-26 21:29:15.095755416 +0000 UTC m=+0.527179268 container remove e4c55b05fe60a3bfc9eab5e7150544ecfeaa77bd800c2c247429cd3aceed9a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T21:29:15.690 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:15 smithi171 podman[69503]: 2024-04-26 21:29:15.331459584 +0000 UTC m=+0.029438101 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:29:15.690 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:15 smithi171 podman[69503]: 2024-04-26 21:29:15.437861541 +0000 UTC m=+0.135840064 container create 46927c18479495ce1838b0acac29d43d53aa62e38c2255e58474cf53e4921463 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:29:15.988 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:15 smithi171 podman[69503]: 2024-04-26 21:29:15.69014079 +0000 UTC m=+0.388119307 container init 46927c18479495ce1838b0acac29d43d53aa62e38c2255e58474cf53e4921463 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T21:29:15.988 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:15 smithi171 podman[69503]: 2024-04-26 21:29:15.693242354 +0000 UTC m=+0.391220873 container start 46927c18479495ce1838b0acac29d43d53aa62e38c2255e58474cf53e4921463 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T21:29:15.988 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:15 smithi171 podman[69503]: 2024-04-26 21:29:15.773794941 +0000 UTC m=+0.471773468 container attach 46927c18479495ce1838b0acac29d43d53aa62e38c2255e58474cf53e4921463 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T21:29:16.241 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 26 21:29:15 smithi171 podman[69503]: 2024-04-26 21:29:15.988369323 +0000 UTC m=+0.686347850 container died 46927c18479495ce1838b0acac29d43d53aa62e38c2255e58474cf53e4921463 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T21:29:16.571 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T21:29:16.610 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:16.610 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T21:29:16.610 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T21:29:16.610 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.a 2024-04-26T21:29:16.907 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:29:16 smithi129 systemd[1]: Stopping Ceph mds.a for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:17.219 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:29:16 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a[71023]: 2024-04-26T21:29:16.905+0000 7f2b54af0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:17.219 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:29:16 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a[71023]: 2024-04-26T21:29:16.905+0000 7f2b54af0700 -1 mds.a *** got signal Terminated *** 2024-04-26T21:29:20.372 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:29:20 smithi129 podman[92436]: 2024-04-26 21:29:20.040269707 +0000 UTC m=+3.237616869 container died 244246ed44dad71cf235050e08a7649f6bfb046c1b79dfe2320ddefeaca3885b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T21:29:20.372 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:29:20 smithi129 podman[92436]: 2024-04-26 21:29:20.372117122 +0000 UTC m=+3.569464258 container cleanup 244246ed44dad71cf235050e08a7649f6bfb046c1b79dfe2320ddefeaca3885b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T21:29:20.720 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:29:20 smithi129 bash[92436]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a 2024-04-26T21:29:20.720 INFO:journalctl@ceph.mds.a.smithi129.stdout:Apr 26 21:29:20 smithi129 podman[92450]: 2024-04-26 21:29:20.581730406 +0000 UTC m=+0.536590219 container remove 244246ed44dad71cf235050e08a7649f6bfb046c1b79dfe2320ddefeaca3885b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:29:20.804 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T21:29:20.837 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:20.837 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T21:29:20.837 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T21:29:20.837 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.d 2024-04-26T21:29:21.168 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:29:20 smithi129 systemd[1]: Stopping Ceph mds.d for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:21.470 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:29:21 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-d[70015]: 2024-04-26T21:29:21.167+0000 7f39ea0d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:21.470 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:29:21 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-d[70015]: 2024-04-26T21:29:21.167+0000 7f39ea0d7700 -1 mds.d *** got signal Terminated *** 2024-04-26T21:29:23.860 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:29:23 smithi129 podman[92558]: 2024-04-26 21:29:23.586460961 +0000 UTC m=+2.527105300 container died 5891d7cd780f8e2615b0e37b5e2d377ae3a22cb024b8592f1e9805960f085f3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T21:29:24.210 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:29:23 smithi129 podman[92558]: 2024-04-26 21:29:23.950920675 +0000 UTC m=+2.891565001 container cleanup 5891d7cd780f8e2615b0e37b5e2d377ae3a22cb024b8592f1e9805960f085f3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T21:29:24.210 INFO:journalctl@ceph.mds.d.smithi129.stdout:Apr 26 21:29:23 smithi129 bash[92558]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-d 2024-04-26T21:29:24.377 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T21:29:24.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:24.409 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T21:29:24.409 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T21:29:24.409 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.g 2024-04-26T21:29:24.719 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:29:24 smithi129 systemd[1]: Stopping Ceph mds.g for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:25.219 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:29:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g[70618]: 2024-04-26T21:29:24.729+0000 7f4fe3e0f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:25.220 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:29:24 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g[70618]: 2024-04-26T21:29:24.729+0000 7f4fe3e0f700 -1 mds.g *** got signal Terminated *** 2024-04-26T21:29:29.188 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:29:28 smithi129 podman[92680]: 2024-04-26 21:29:28.74103346 +0000 UTC m=+4.115816276 container died 74fb377ae3ac27128fa75581cf0b896f169ac018c74f9902416367ef46b5c85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T21:29:29.470 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:29:29 smithi129 podman[92680]: 2024-04-26 21:29:29.270730752 +0000 UTC m=+4.645513556 container cleanup 74fb377ae3ac27128fa75581cf0b896f169ac018c74f9902416367ef46b5c85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T21:29:29.470 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:29:29 smithi129 bash[92680]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g 2024-04-26T21:29:29.739 INFO:journalctl@ceph.mds.g.smithi129.stdout:Apr 26 21:29:29 smithi129 podman[92694]: 2024-04-26 21:29:29.488631138 +0000 UTC m=+0.747856938 container remove 74fb377ae3ac27128fa75581cf0b896f169ac018c74f9902416367ef46b5c85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T21:29:29.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T21:29:29.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:29.773 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T21:29:29.774 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T21:29:29.774 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.j 2024-04-26T21:29:30.083 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:29:29 smithi129 systemd[1]: Stopping Ceph mds.j for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:30.469 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:29:30 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-j[69667]: 2024-04-26T21:29:30.081+0000 7fec82bbb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:30.470 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:29:30 smithi129 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-j[69667]: 2024-04-26T21:29:30.081+0000 7fec82bbb700 -1 mds.j *** got signal Terminated *** 2024-04-26T21:29:32.791 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:29:32 smithi129 podman[92800]: 2024-04-26 21:29:32.531255681 +0000 UTC m=+2.547198340 container died 5187b1d1621778550ddf546935950e1385fcae05fbcc2feb6158c43606540867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T21:29:33.150 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:29:32 smithi129 podman[92800]: 2024-04-26 21:29:32.891024728 +0000 UTC m=+2.906967385 container cleanup 5187b1d1621778550ddf546935950e1385fcae05fbcc2feb6158c43606540867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T21:29:33.150 INFO:journalctl@ceph.mds.j.smithi129.stdout:Apr 26 21:29:32 smithi129 bash[92800]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-j 2024-04-26T21:29:33.322 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T21:29:33.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:33.352 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T21:29:33.352 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T21:29:33.353 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.b 2024-04-26T21:29:33.634 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:29:33 smithi150 systemd[1]: Stopping Ceph mds.b for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:34.132 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:29:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b[62955]: 2024-04-26T21:29:33.686+0000 7f6c48470700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:34.133 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:29:33 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b[62955]: 2024-04-26T21:29:33.686+0000 7f6c48470700 -1 mds.b *** got signal Terminated *** 2024-04-26T21:29:36.937 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:29:36 smithi150 podman[67671]: 2024-04-26 21:29:36.655815391 +0000 UTC m=+3.109609574 container died d3dd5bd2c80934d401ed39cd447180a81a868a563567695d0f0fd9e072b41554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T21:29:37.312 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:29:37 smithi150 podman[67671]: 2024-04-26 21:29:37.01960232 +0000 UTC m=+3.473396500 container cleanup d3dd5bd2c80934d401ed39cd447180a81a868a563567695d0f0fd9e072b41554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T21:29:37.312 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 26 21:29:37 smithi150 bash[67671]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-b 2024-04-26T21:29:37.413 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T21:29:37.448 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:37.448 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T21:29:37.448 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T21:29:37.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.e 2024-04-26T21:29:37.632 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 26 21:29:37 smithi150 systemd[1]: Stopping Ceph mds.e for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:38.132 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 26 21:29:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-e[62336]: 2024-04-26T21:29:37.763+0000 7fdea4302700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:38.133 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 26 21:29:37 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-e[62336]: 2024-04-26T21:29:37.763+0000 7fdea4302700 -1 mds.e *** got signal Terminated *** 2024-04-26T21:29:42.050 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 26 21:29:41 smithi150 podman[67805]: 2024-04-26 21:29:41.782726445 +0000 UTC m=+4.118228266 container died 066ee3c811ea2473a2a8d03d8036950af0bcbf2666dcf497060943955881c8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T21:29:42.383 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 26 21:29:42 smithi150 podman[67805]: 2024-04-26 21:29:42.140447522 +0000 UTC m=+4.475949350 container cleanup 066ee3c811ea2473a2a8d03d8036950af0bcbf2666dcf497060943955881c8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS) 2024-04-26T21:29:42.383 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 26 21:29:42 smithi150 bash[67805]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-e 2024-04-26T21:29:42.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T21:29:42.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:42.601 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T21:29:42.601 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T21:29:42.601 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.h 2024-04-26T21:29:42.882 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:29:42 smithi150 systemd[1]: Stopping Ceph mds.h for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:43.382 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:29:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-h[62648]: 2024-04-26T21:29:42.935+0000 7f5aa6a3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:43.383 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:29:42 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-h[62648]: 2024-04-26T21:29:42.935+0000 7f5aa6a3c700 -1 mds.h *** got signal Terminated *** 2024-04-26T21:29:47.237 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:29:46 smithi150 podman[67925]: 2024-04-26 21:29:46.947485669 +0000 UTC m=+4.135747812 container died 0c77778bd39642c1e37cbcf8c7f839dfa85458c8e83bb3201a59e10e0981b2bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T21:29:47.579 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:29:47 smithi150 podman[67925]: 2024-04-26 21:29:47.328030323 +0000 UTC m=+4.516292456 container cleanup 0c77778bd39642c1e37cbcf8c7f839dfa85458c8e83bb3201a59e10e0981b2bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T21:29:47.579 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 26 21:29:47 smithi150 bash[67925]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-h 2024-04-26T21:29:47.712 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T21:29:47.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:47.747 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T21:29:47.748 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T21:29:47.748 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.k 2024-04-26T21:29:47.882 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:47 smithi150 systemd[1]: Stopping Ceph mds.k for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:48.382 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k[62019]: 2024-04-26T21:29:48.055+0000 7f2d4b48d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:48.383 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:48 smithi150 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k[62019]: 2024-04-26T21:29:48.055+0000 7f2d4b48d700 -1 mds.k *** got signal Terminated *** 2024-04-26T21:29:49.882 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:49 smithi150 podman[68045]: 2024-04-26 21:29:49.593924768 +0000 UTC m=+1.636826504 container died 7d6369958e722e270a69c661c9e79ae5230ab1648c62c8e3219203623f2d6936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:29:50.161 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:49 smithi150 podman[68045]: 2024-04-26 21:29:49.965314071 +0000 UTC m=+2.008215808 container cleanup 7d6369958e722e270a69c661c9e79ae5230ab1648c62c8e3219203623f2d6936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T21:29:50.161 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:49 smithi150 bash[68045]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k 2024-04-26T21:29:50.389 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T21:29:50.411 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:50 smithi150 podman[68059]: 2024-04-26 21:29:50.2662705 +0000 UTC m=+0.658564718 container remove 7d6369958e722e270a69c661c9e79ae5230ab1648c62c8e3219203623f2d6936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T21:29:50.411 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:50 smithi150 systemd[1]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-26T21:29:50.411 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:50 smithi150 systemd[1]: Stopped Ceph mds.k for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:29:50.411 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 26 21:29:50 smithi150 systemd[1]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.000s CPU time. 2024-04-26T21:29:50.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:50.420 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T21:29:50.420 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T21:29:50.420 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.c 2024-04-26T21:29:50.718 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:29:50 smithi171 systemd[1]: Stopping Ceph mds.c for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:50.977 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:29:50 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-c[64165]: 2024-04-26T21:29:50.715+0000 7f17cd6f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:50.978 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:29:50 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-c[64165]: 2024-04-26T21:29:50.715+0000 7f17cd6f3700 -1 mds.c *** got signal Terminated *** 2024-04-26T21:29:52.734 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:29:52 smithi171 podman[69608]: 2024-04-26 21:29:52.346137554 +0000 UTC m=+1.737633438 container died c058ac99f18fa4e342536a68d6e86bc8276053d211c02cee6e20cb3699e77774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T21:29:52.991 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:29:52 smithi171 podman[69608]: 2024-04-26 21:29:52.833413062 +0000 UTC m=+2.224908938 container cleanup c058ac99f18fa4e342536a68d6e86bc8276053d211c02cee6e20cb3699e77774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T21:29:52.991 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 26 21:29:52 smithi171 bash[69608]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-c 2024-04-26T21:29:53.263 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T21:29:53.298 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:53.298 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T21:29:53.298 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T21:29:53.299 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.f 2024-04-26T21:29:53.724 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:53 smithi171 systemd[1]: Stopping Ceph mds.f for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:53.991 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:53 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-f[63862]: 2024-04-26T21:29:53.722+0000 7f35bdd7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:53.991 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:53 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-f[63862]: 2024-04-26T21:29:53.722+0000 7f35bdd7f700 -1 mds.f *** got signal Terminated *** 2024-04-26T21:29:57.716 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:57 smithi171 podman[69741]: 2024-04-26 21:29:57.443038723 +0000 UTC m=+3.912028637 container died 93fd21c112dd6f8eefc6cd09502c28444b7c6bbfc9689b01b52caa9f6a95023a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T21:29:57.977 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:57 smithi171 podman[69741]: 2024-04-26 21:29:57.789768056 +0000 UTC m=+4.258757971 container cleanup 93fd21c112dd6f8eefc6cd09502c28444b7c6bbfc9689b01b52caa9f6a95023a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T21:29:57.978 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:57 smithi171 bash[69741]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-f 2024-04-26T21:29:58.218 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T21:29:58.238 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:57 smithi171 podman[69755]: 2024-04-26 21:29:57.974528684 +0000 UTC m=+0.527147384 container remove 93fd21c112dd6f8eefc6cd09502c28444b7c6bbfc9689b01b52caa9f6a95023a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:29:58.238 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:58 smithi171 systemd[1]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-26T21:29:58.238 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:58 smithi171 systemd[1]: Stopped Ceph mds.f for eccca2b2-0410-11ef-bc93-c7b262605968. 2024-04-26T21:29:58.238 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 26 21:29:58 smithi171 systemd[1]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.f.service: Consumed 1.020s CPU time. 2024-04-26T21:29:58.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:29:58.251 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T21:29:58.251 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T21:29:58.251 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.i 2024-04-26T21:29:58.491 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:29:58 smithi171 systemd[1]: Stopping Ceph mds.i for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:29:58.991 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:29:58 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i[64475]: 2024-04-26T21:29:58.584+0000 7f2ba4107700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:29:58.991 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:29:58 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i[64475]: 2024-04-26T21:29:58.584+0000 7f2ba4107700 -1 mds.i *** got signal Terminated *** 2024-04-26T21:30:00.937 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:30:00 smithi171 podman[69862]: 2024-04-26 21:30:00.614271579 +0000 UTC m=+2.151879982 container died 2f80c7340542002418e51c936aeb3ea97a56facb7803a86c9dcf58a70c71645c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T21:30:01.241 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:30:01 smithi171 podman[69862]: 2024-04-26 21:30:01.011458 +0000 UTC m=+2.549066429 container cleanup 2f80c7340542002418e51c936aeb3ea97a56facb7803a86c9dcf58a70c71645c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T21:30:01.241 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 26 21:30:01 smithi171 bash[69862]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-i 2024-04-26T21:30:01.387 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T21:30:01.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:30:01.422 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T21:30:01.422 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T21:30:01.423 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.l 2024-04-26T21:30:01.741 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:30:01 smithi171 systemd[1]: Stopping Ceph mds.l for eccca2b2-0410-11ef-bc93-c7b262605968... 2024-04-26T21:30:02.241 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:30:01 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-l[63542]: 2024-04-26T21:30:01.764+0000 7f4c9a7da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:30:02.241 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:30:01 smithi171 ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-l[63542]: 2024-04-26T21:30:01.764+0000 7f4c9a7da700 -1 mds.l *** got signal Terminated *** 2024-04-26T21:30:06.116 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:30:05 smithi171 podman[69983]: 2024-04-26 21:30:05.779625521 +0000 UTC m=+4.141507435 container died ed733b64450ce881beaeedb48bc21b45d9c017c776193eba6e419fce96f6ab82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:30:06.400 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:30:06 smithi171 podman[69983]: 2024-04-26 21:30:06.182116828 +0000 UTC m=+4.543998742 container cleanup ed733b64450ce881beaeedb48bc21b45d9c017c776193eba6e419fce96f6ab82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T21:30:06.400 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 26 21:30:06 smithi171 bash[69983]: ceph-eccca2b2-0410-11ef-bc93-c7b262605968-mds-l 2024-04-26T21:30:06.620 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eccca2b2-0410-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T21:30:06.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:30:06.651 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T21:30:06.651 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eccca2b2-0410-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T21:30:06.844 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:30:09.234 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eccca2b2-0410-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T21:30:09.426 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:30:11.946 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eccca2b2-0410-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T21:30:12.137 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:30:14.421 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:30:14.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:30:14.483 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:30:14.509 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T21:30:14.511 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/remote/smithi129/crash 2024-04-26T21:30:14.511 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/crash -- . 2024-04-26T21:30:14.539 INFO:teuthology.orchestra.run.smithi129.stderr:tar: /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T21:30:14.539 INFO:teuthology.orchestra.run.smithi129.stderr:tar: Error is not recoverable: exiting now 2024-04-26T21:30:14.541 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/remote/smithi150/crash 2024-04-26T21:30:14.542 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/crash -- . 2024-04-26T21:30:14.603 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T21:30:14.603 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-04-26T21:30:14.605 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/remote/smithi171/crash 2024-04-26T21:30:14.606 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/crash -- . 2024-04-26T21:30:14.650 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T21:30:14.650 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-04-26T21:30:14.652 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T21:30:14.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-26T21:30:14.691 INFO:tasks.cephadm:Compressing logs... 2024-04-26T21:30:14.692 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:30:14.734 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:30:14.737 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:30:14.758 INFO:teuthology.orchestra.run.smithi129.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T21:30:14.758 INFO:teuthology.orchestra.run.smithi129.stderr:: No such file or directory 2024-04-26T21:30:14.758 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log 2024-04-26T21:30:14.758 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T21:30:14.759 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T21:30:14.759 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T21:30:14.760 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T21:30:14.763 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T21:30:14.764 INFO:teuthology.orchestra.run.smithi171.stderr:: No such file or directory 2024-04-26T21:30:14.764 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log 2024-04-26T21:30:14.764 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T21:30:14.764 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T21:30:14.764 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T21:30:14.764 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T21:30:14.764 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.audit.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T21:30:14.764 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T21:30:14.764 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mgr.z.log: 86.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr: 94.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr: 94.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T21:30:14.772 INFO:teuthology.orchestra.run.smithi171.stderr: 82.5% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T21:30:14.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T21:30:14.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T21:30:14.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-volume.log: 92.8% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mgr.x.log: 86.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stderr: 90.3% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stderr: 94.8% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.cephadm.log: 94.8% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stderr: 82.9% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-volume.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stderr: 94.8% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T21:30:14.780 INFO:teuthology.orchestra.run.smithi129.stderr: 91.1% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T21:30:14.781 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T21:30:14.785 INFO:teuthology.orchestra.run.smithi171.stderr: 94.7% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T21:30:14.788 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T21:30:14.791 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T21:30:14.791 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log 2024-04-26T21:30:14.791 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T21:30:14.791 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T21:30:14.791 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T21:30:14.791 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T21:30:14.791 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T21:30:14.794 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T21:30:14.794 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T21:30:14.794 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T21:30:14.794 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T21:30:14.794 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T21:30:14.794 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T21:30:14.794 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T21:30:14.794 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T21:30:14.954 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T21:30:14.973 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T21:30:14.978 INFO:teuthology.orchestra.run.smithi150.stderr: 82.5% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T21:30:14.995 INFO:teuthology.orchestra.run.smithi150.stderr: 94.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T21:30:15.000 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T21:30:15.017 INFO:teuthology.orchestra.run.smithi150.stderr: 94.7% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T21:30:15.020 INFO:teuthology.orchestra.run.smithi150.stderr: 94.7% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T21:30:15.030 INFO:teuthology.orchestra.run.smithi129.stderr: 90.4% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T21:30:15.155 INFO:teuthology.orchestra.run.smithi171.stderr: 91.9% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T21:30:15.161 INFO:teuthology.orchestra.run.smithi150.stderr: 90.4% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T21:30:15.257 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T21:30:15.558 INFO:teuthology.orchestra.run.smithi129.stderr: 90.5% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T21:30:18.212 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T21:30:23.277 INFO:teuthology.orchestra.run.smithi129.stderr: 92.1% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T21:30:24.850 INFO:teuthology.orchestra.run.smithi171.stderr: 91.5% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stderr: 91.7% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stderr: 91.4% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T21:30:26.098 INFO:teuthology.orchestra.run.smithi129.stderr: 91.5% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T21:30:27.755 INFO:teuthology.orchestra.run.smithi129.stderr: 91.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T21:30:27.909 INFO:teuthology.orchestra.run.smithi129.stderr: 91.7% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T21:30:27.911 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-26T21:30:27.911 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m13.163s 2024-04-26T21:30:27.911 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m45.524s 2024-04-26T21:30:27.911 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m1.511s 2024-04-26T21:30:29.611 INFO:teuthology.orchestra.run.smithi171.stderr: 91.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T21:30:36.341 INFO:teuthology.orchestra.run.smithi171.stderr: 94.8% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T21:30:36.343 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-04-26T21:30:36.343 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m21.592s 2024-04-26T21:30:36.343 INFO:teuthology.orchestra.run.smithi171.stderr:user 1m6.092s 2024-04-26T21:30:36.343 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m1.972s 2024-04-26T21:31:34.130 INFO:teuthology.orchestra.run.smithi150.stderr: 91.5% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T21:32:00.791 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T21:32:08.501 INFO:teuthology.orchestra.run.smithi150.stderr: 91.6% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T21:32:15.976 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T21:36:32.032 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/eccca2b2-0410-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T21:36:32.034 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-26T21:36:32.034 INFO:teuthology.orchestra.run.smithi150.stderr:real 6m17.282s 2024-04-26T21:36:32.034 INFO:teuthology.orchestra.run.smithi150.stderr:user 4m51.565s 2024-04-26T21:36:32.034 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m10.400s 2024-04-26T21:36:32.034 INFO:tasks.cephadm:Archiving logs... 2024-04-26T21:36:32.035 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/remote/smithi129/log 2024-04-26T21:36:32.036 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:36:39.852 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/remote/smithi150/log 2024-04-26T21:36:39.853 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:37:30.249 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/remote/smithi171/log 2024-04-26T21:37:30.250 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:37:40.141 INFO:tasks.cephadm:Removing cluster... 2024-04-26T21:37:40.142 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eccca2b2-0410-11ef-bc93-c7b262605968 --force 2024-04-26T21:37:40.333 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:37:40.735 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eccca2b2-0410-11ef-bc93-c7b262605968 --force 2024-04-26T21:37:41.058 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:37:41.577 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eccca2b2-0410-11ef-bc93-c7b262605968 --force 2024-04-26T21:37:41.767 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: eccca2b2-0410-11ef-bc93-c7b262605968 2024-04-26T21:37:42.151 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T21:37:42.151 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T21:37:42.168 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T21:37:42.184 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T21:37:42.199 INFO:tasks.cephadm:Teardown complete 2024-04-26T21:37:42.200 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T21:37:42.210 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T21:37:42.210 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:37:42.213 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:37:42.227 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:37:42.339 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T21:37:42.350 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T21:37:42.350 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:37:42.353 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:37:42.355 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:37:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:37:42.367 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:37:42.369 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:37:42.370 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:37:42.370 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-04-26T21:37:42.370 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 +1148us[+1164us] +/- 40ms 2024-04-26T21:37:42.370 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -884us[ -868us] +/- 72ms 2024-04-26T21:37:42.370 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -1300us[-1300us] +/- 88ms 2024-04-26T21:37:42.370 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:42.373 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:37:42.373 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-26T21:37:42.373 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +1191us[+1201us] +/- 40ms 2024-04-26T21:37:42.373 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -836us[ -836us] +/- 72ms 2024-04-26T21:37:42.373 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 -1255us[-1255us] +/- 88ms 2024-04-26T21:37:42.373 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:42.399 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:37:42.399 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-26T21:37:42.399 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +1124us[+1133us] +/- 40ms 2024-04-26T21:37:42.399 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -894us[ -894us] +/- 72ms 2024-04-26T21:37:42.399 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -1321us[-1321us] +/- 88ms 2024-04-26T21:37:42.399 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:42.401 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T21:37:42.421 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T21:37:42.421 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T21:37:42.454 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:37:42.527 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:37:42.609 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:37:42.673 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:37:42.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:37:42.702 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:37:42.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:37:42.729 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:37:42.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:37:42.756 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T21:37:42.767 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T21:37:42.816 INFO:teuthology.task.internal:Duration was 2254.135809 seconds 2024-04-26T21:37:42.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T21:37:42.849 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T21:37:42.849 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:37:42.852 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:37:42.854 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:37:42.886 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:37:42.890 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:37:42.917 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:37:43.285 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T21:37:43.285 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-04-26T21:37:43.285 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:37:43.319 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-04-26T21:37:43.319 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:37:43.348 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-04-26T21:37:43.348 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:37:43.371 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T21:37:43.371 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:37:43.374 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:37:43.391 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:37:43.435 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T21:37:43.435 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:37:43.439 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:37:43.457 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:37:43.814 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T21:37:43.826 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T21:37:43.826 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:37:43.850 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:37:43.875 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:37:43.900 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T21:37:43.911 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:37:43.914 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:37:43.918 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:37:43.933 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-04-26T21:37:43.946 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-04-26T21:37:43.961 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-04-26T21:37:43.993 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:37:44.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:37:44.007 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:37:44.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:37:44.022 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:37:44.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:37:44.047 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T21:37:44.057 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T21:37:44.058 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/remote/smithi129 2024-04-26T21:37:44.058 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:37:44.099 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/remote/smithi150 2024-04-26T21:37:44.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:37:44.139 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675202/remote/smithi171 2024-04-26T21:37:44.139 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:37:44.177 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T21:37:44.178 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:37:44.181 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:37:44.183 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:37:44.224 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T21:37:44.238 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T21:37:44.238 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T21:37:44.275 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T21:37:44.276 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:37:44.278 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:37:44.280 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:37:44.290 INFO:teuthology.orchestra.run.smithi129.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:37 /home/ubuntu/cephtest 2024-04-26T21:37:44.292 INFO:teuthology.orchestra.run.smithi150.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:37 /home/ubuntu/cephtest 2024-04-26T21:37:44.294 INFO:teuthology.orchestra.run.smithi171.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:37 /home/ubuntu/cephtest 2024-04-26T21:37:44.299 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T21:37:44.311 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T21:37:44.378 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} duration: 2254.1358087062836 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T21:37:44.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:37:44.456 INFO:teuthology.run:pass